-- Logs begin at Mon 2022-08-29 19:43:51 UTC, end at Mon 2022-08-29 20:53:58 UTC. -- Aug 29 20:18:25.514490 dv-cinder-854263-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 29 20:18:25.523661 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 29 20:18:25.526694 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 29 20:18:25 2022] *** Aug 29 20:18:25.526846 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 29 20:18:25.526956 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 29 20:18:25.527066 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: nodename: dv-cinder-854263-2-iscsi Aug 29 20:18:25.527203 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: machine: x86_64 Aug 29 20:18:25.527311 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: clock source: unix Aug 29 20:18:25.527405 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: pcre jit disabled Aug 29 20:18:25.527522 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: detected number of CPU cores: 4 Aug 29 20:18:25.527646 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: current working directory: / Aug 29 20:18:25.527747 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: detected binary path: /usr/bin/uwsgi-core Aug 29 20:18:25.527848 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: your processes number limit is 47680 Aug 29 20:18:25.527967 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: your memory page size is 4096 bytes Aug 29 20:18:25.528080 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: detected max file descriptor number: 2048 Aug 29 20:18:25.528190 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: lock engine: pthread robust mutexes Aug 29 20:18:25.528510 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: thunder lock: enabled Aug 29 20:18:25.528814 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 29 20:18:25.529165 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 29 20:18:25.569472 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: Python main interpreter initialized at 0x55c0f5cec9c0 Aug 29 20:18:25.569472 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: python threads support enabled Aug 29 20:18:25.569472 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: your server socket listen backlog is limited to 100 connections Aug 29 20:18:25.569472 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: your mercy for graceful operations on workers is 90 seconds Aug 29 20:18:25.569899 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: mapped 403077 bytes (393 KB) for 2 cores Aug 29 20:18:25.569963 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: *** Operational MODE: preforking *** Aug 29 20:18:25.570239 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: *** uWSGI is running in multiple interpreter mode *** Aug 29 20:18:25.570239 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: spawned uWSGI master process (pid: 81388) Aug 29 20:18:25.570456 dv-cinder-854263-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 29 20:18:25.576093 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: spawned uWSGI worker 1 (pid: 81397, cores: 1) Aug 29 20:18:25.576093 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: spawned uWSGI worker 2 (pid: 81398, cores: 1) Aug 29 20:18:25.576093 dv-cinder-854263-2-iscsi devstack@keystone.service[81388]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 29 20:18:29.934704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 20:18:29.934704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: from cryptography.utils import int_from_bytes Aug 29 20:18:29.935517 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 20:18:29.935517 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: from cryptography.utils import int_from_bytes Aug 29 20:18:29.985095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 20:18:29.985280 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 20:18:29.985457 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] command line args: [] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 20:18:29.985719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 20:18:29.985893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ================================================================================ {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 20:18:29.986139 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] admin_token = **** {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.986380 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] config_dir = [] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.986569 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.986778 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] config_source = [] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.986922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] debug = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.987115 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.987480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.987655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.987899 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.988172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.988397 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] list_limit = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.988574 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] log_config_append = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.988748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.989443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] log_dir = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.989635 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] log_file = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.989855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.990098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.990336 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.990576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.990752 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.990926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.991165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.991415 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.991633 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.991860 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.992114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.992353 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.992587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.992825 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.993107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.993576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.993755 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] publish_errors = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.993984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.994210 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.994385 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.994623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.994852 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.995086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] standard_threads = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.995316 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.995548 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.995719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.995949 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] use_journal = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.996177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] use_json = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.996273 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] use_stderr = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.996398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] use_syslog = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.996628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:29.996876 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:29.997109 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:29.997420 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:29.997592 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:29.997835 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:29.998068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:29.998306 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:29.998536 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:29.998796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:29.999016 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:29.999265 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:29.999490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:29.999732 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:29.999987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.000221 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.000461 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.000744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.000939 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.001190 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.001492 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.001659 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.001911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.002165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.002414 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.002655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.002878 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.003102 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.003328 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.003557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.003798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.004074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.004304 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.004529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.005331 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.005572 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.005914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.006045 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.006271 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.006503 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.006786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.006958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.007185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.007418 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.007658 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.007860 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.008131 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.008355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.008599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.008808 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.009042 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.009268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.009513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.009736 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.009968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.010201 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.010429 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.010661 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.010893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.011115 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.011357 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.011636 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.011823 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.012072 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.012335 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.012566 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.012794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.013024 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.013253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.013516 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.013796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.013992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.014236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.014505 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.014699 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.014933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.015175 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.015421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.015638 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.015888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.016157 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.016403 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.016633 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.016864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.017099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.017326 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.017556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.017842 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.018051 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.018293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.018533 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.018816 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.019000 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.019235 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.019462 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.019695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.019917 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.020188 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.020414 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.020642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.020896 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.021103 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.021335 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.021573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.021807 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.022053 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.022345 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.022550 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.022791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] auth.external = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.023002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.023292 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.023521 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.023742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] auth.password = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.023994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] auth.token = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.024230 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.024461 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.024703 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.024931 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.025202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.025413 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.025642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.025872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] credential.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.026114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.026353 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.026597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.026831 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.027067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.027295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.027582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.027797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.028037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.028550 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.028829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.029069 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.029327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.029570 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.029874 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.030043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] federation.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.030043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.030514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.030760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.030935 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.031176 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.031640 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.031640 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.031868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.032421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.032668 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.032895 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.033126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.033369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.033612 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.033841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.034072 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.034314 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.034538 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.034766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.035031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.035245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.035476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.035697 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.035983 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.036377 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.036507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.036680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.036926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.037210 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.037429 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.037659 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.037951 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.038115 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.038344 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.038569 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.038797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.039035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.039267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.039510 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.039734 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.039992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.040234 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.040471 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.040710 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.040951 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.041140 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 20:18:30.041140 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: from cryptography.utils import int_from_bytes Aug 29 20:18:30.041800 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.041800 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.041939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 29 20:18:30.041939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: from cryptography.utils import int_from_bytes Aug 29 20:18:30.042041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.042041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.042041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.042373 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.042563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.042734 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.042974 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.043210 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.043440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.043703 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.043901 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.044201 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.044400 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.044632 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.044862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.045088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.045322 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.045574 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.045801 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.046061 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.046277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.046518 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.046743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.046973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.047198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.047424 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.047658 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.047880 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.048228 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.048475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.048685 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.048809 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.049046 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.049270 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.049556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.049756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.049994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.050156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.050384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.050622 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.050905 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.051090 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.051388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.052795 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.052795 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.052795 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.052795 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.053558 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.053558 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.053831 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.053831 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.054241 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.054241 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] resource.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.054658 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.054942 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.054942 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.055212 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.055394 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.055623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.055866 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.056121 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.056357 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.056590 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] role.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.056830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] role.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.057055 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.057299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.057541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.057783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.058023 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.058263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.058504 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.058743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.058983 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.059208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.059449 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.059689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.059945 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.060186 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.060644 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.060877 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.061139 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.061382 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.061621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.061827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.062010 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.062265 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.062512 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.062798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.062961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.063259 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.063386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.063634 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.063882 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.064191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.064380 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.064613 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.064838 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.065070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] token.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.065301 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.065544 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.065775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.066023 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.066264 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.066499 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.066739 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.066977 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.067811 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.068068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.068304 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.068535 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.068776 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.069051 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.069276 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.069514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.069776 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.070032 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.connection = **** {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.070268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.070509 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.070742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.070978 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.071209 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.071440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.071670 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.071898 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.072242 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.072475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.072712 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.073245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.073498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.073730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.073971 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.074195 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.074431 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.074660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.074826 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81398) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 20:18:30.095398 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 29 20:18:30.095580 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 29 20:18:30.095767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] command line args: [] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 29 20:18:30.096073 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 29 20:18:30.096252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ================================================================================ {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 29 20:18:30.096509 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] admin_token = **** {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.096759 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] config_dir = [] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.096946 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.097135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] config_source = [] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.097355 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] debug = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.097558 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.097776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.097957 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.098218 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.099080 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.099326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] list_limit = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.099510 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] log_config_append = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.099686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.099864 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] log_dir = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.100063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] log_file = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.100303 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.100572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.100819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.101072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.101255 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.101431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.101679 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.101949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.102180 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.102411 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.102670 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.102907 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.103175 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.103445 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.103745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.104305 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.104502 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] publish_errors = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.104737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.104834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.104971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.105058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.105142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.105250 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.105386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.105623 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.105859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] standard_threads = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.106093 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.106336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.106537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.106712 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] use_journal = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.106890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] use_json = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.107065 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] use_stderr = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.107240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] use_syslog = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.107472 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 29 20:18:30.107728 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.107980 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.108267 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.108503 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.108785 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.109046 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.109387 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.109708 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.109979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.110238 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.110552 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.110800 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.111267 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.111420 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.111692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.111979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.112227 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.112616 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.113003 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.113230 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.113473 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.113825 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.114081 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.114995 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.115315 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.115535 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.115731 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.115996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.116334 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.116715 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.116897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.117153 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.117459 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.117708 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.118083 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.118357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.118607 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.118747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.118874 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.119067 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.119292 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.119523 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.119754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.120006 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.120240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.120478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.120728 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.120961 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.121199 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.121429 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.121665 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.121907 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.122182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.122409 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.122646 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.122881 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.123118 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.123354 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.123590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.123846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.124118 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.124355 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.124596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.124831 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.125092 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.125329 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.125561 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.125799 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.126036 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.126308 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.126546 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.126795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.127044 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.127336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.127545 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.127782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.128061 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.128338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.128575 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.129224 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.129224 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.129455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.129754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.130092 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.130418 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.130586 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.130849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.131162 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.131506 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.131757 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.132058 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.132321 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.132567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.132896 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.133174 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.133434 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.133665 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.134123 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.134341 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.134604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.134833 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.135119 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.135471 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.135717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.135983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.136259 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.136680 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.136913 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.137143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] auth.external = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.137353 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.137827 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.138046 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.138248 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] auth.password = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.138486 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] auth.token = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.138752 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.139098 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.139431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.139609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.139879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.140286 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.140511 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.140755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] credential.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.141027 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.141470 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.141711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.141976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.142243 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.142668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.142907 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.143167 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.143470 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.143818 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.144518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.144518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.144901 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.145216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.145399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.145664 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] federation.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.145996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.146318 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.146532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.146793 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.147044 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.147346 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.147655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.147922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.148203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.148487 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.148819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.149070 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.149331 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.149602 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.149862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.150210 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.150469 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.150735 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.150978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.151277 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.151605 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.151862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.152123 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.152378 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.152730 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.153052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.153567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.153828 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.154100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.154470 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.154730 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.154866 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.155000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.155121 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.155249 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.155540 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.155782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.156068 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.156338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.156640 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.156947 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.157205 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.157457 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.157907 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.158168 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.158464 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.158718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.159005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.159452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.159721 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.160029 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.160307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.161267 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.161552 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.162059 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.162469 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.162630 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.163032 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.163278 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.163553 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.163853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.164098 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.164418 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.164694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.164931 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.165187 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.165469 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.165793 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.166060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.166337 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.166587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.167003 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.167266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.167507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.167779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.168153 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.168421 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.168693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.168951 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.169368 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.169638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.169933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.170292 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.170564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.170810 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.171072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.171780 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.172079 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.172332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.172614 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.172998 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.173314 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.173570 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.173821 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.174083 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.174342 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.174727 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.175217 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.175487 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.175710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.175970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.176469 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] resource.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.177227 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.177543 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.177788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.178042 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.178439 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.178716 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.178968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.179210 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.179452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.179692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] role.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.179952 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] role.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.180196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.180472 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.180731 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.181000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.181255 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.181520 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.181775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.182037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.182423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.182702 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.182953 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.183216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.183472 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.183732 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.184363 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.184607 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.184903 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.185136 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.185390 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.185666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.185930 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.186229 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.186587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.186834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.187094 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.187351 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.187608 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.187980 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.188249 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.188501 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.188743 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.188985 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.189220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.189485 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] token.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.189752 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.190005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.190261 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.190540 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.190890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.191138 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.191397 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.191665 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.191958 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.192392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.193142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.193142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.193142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.193545 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.193640 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.193904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.194178 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.194463 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.connection = **** {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.194809 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.195066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.195329 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.195593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.195853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.197026 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.197026 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.197026 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.197026 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.197451 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.197451 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.197627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.197891 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.198149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.198414 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.198711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.198994 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.199259 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 29 20:18:30.199429 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81397) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 29 20:18:30.210785 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.227517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.227820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.228073 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.237352 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.257775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.258005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.262020 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.262271 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.298040 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.334728 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.335038 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.420969 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.421288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.421409 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.421612 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.421827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.422042 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.448730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.448922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.449083 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.449308 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.449525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.449744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.449957 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.462483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.476918 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.477148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.477375 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.477596 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.477813 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.478028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.478242 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81398) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.484967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.485221 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.485447 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.485670 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.485906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.486146 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.496431 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.496645 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 20:18:30.496993 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.497197 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 20:18:30.497475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.497740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.498001 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.498291 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.500334 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.500587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.500813 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.501055 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.501276 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.501513 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.501730 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.508964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.509272 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.509586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.509875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.510362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.513827 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.514056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.514297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.514531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.514752 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.514969 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.515199 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81397) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 29 20:18:30.517308 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.520843 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.521131 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.521418 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.521702 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.521998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.522249 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.522523 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.522844 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.523115 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.523390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.523661 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.526851 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.527059 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 20:18:30.527427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.527625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 29 20:18:30.527922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.528465 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.528750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.529028 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.540241 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.540546 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.540853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.541206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.548641 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.549066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.550196 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.550484 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.554007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.554290 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.554580 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.554880 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.555174 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.555402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.555494 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.555573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.555795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.555892 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.556107 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.556417 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.556717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.557017 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.562547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.562838 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.567066 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.567359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.567647 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.567962 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.568241 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.570333 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.570621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.570909 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.582403 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.584600 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.584967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.587709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.588013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.589940 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.590325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.590614 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.591989 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.595283 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.595581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.598108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.599027 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.600722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.603557 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.603857 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.604031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.604168 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.604478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.604898 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.605188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.605473 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.605878 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.607332 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.607635 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.607925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.608237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.608521 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.608910 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.624811 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.625110 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.625430 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.625723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.625994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.626256 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.629191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.632369 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.632668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.633058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.634720 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.636299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.636579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.637079 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.639514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.639806 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.640126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.640258 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.640428 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.640718 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.643463 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.643789 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.644128 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.644539 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.644824 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.645125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.652553 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.652849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.653389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.653671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.653954 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.654234 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.654529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.661882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.662174 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.662443 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.662709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.669506 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.669946 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.671192 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.672763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.672992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.673078 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.675771 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.675892 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.676083 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.676422 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.677217 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.677511 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.677912 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.679390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.679676 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.679990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.687795 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.689155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.689313 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.689443 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.689724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.690006 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.690288 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.690576 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.690853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.692223 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.692524 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.692818 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.693102 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.701120 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.701365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.701649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.706379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.709135 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81398) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.709423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.709704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.709991 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.710076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.710265 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.710543 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.710819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.711098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.711377 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.711663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.711960 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.712222 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.712486 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.712761 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81398) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.712939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.714407 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.715893 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.716202 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.716485 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.723059 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.725209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.728112 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.728407 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.728694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.728975 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.736973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.737260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.737542 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.744618 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81397) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 29 20:18:30.744907 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.745189 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.745469 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.745748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.746024 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.746312 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.746583 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.746867 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.747137 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.747420 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.747535 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c0f5cec9c0 pid: 81398 (default app) Aug 29 20:18:30.747684 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.747992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.748269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81397) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 29 20:18:30.750984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e802b-d6d6-4559-91ef-97114a5a3722 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:30.751279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e802b-d6d6-4559-91ef-97114a5a3722 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:30.751557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e802b-d6d6-4559-91ef-97114a5a3722 None None] PATH_INFO: `/v3` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:30.755778 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Mon Aug 29 20:18:30 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:18:30.785617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c0f5cec9c0 pid: 81397 (default app) Aug 29 20:18:37.412531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a643a-4c11-4a64-adf8-dfdb7bc8d3a1 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:37.413052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a643a-4c11-4a64-adf8-dfdb7bc8d3a1 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:37.413152 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a643a-4c11-4a64-adf8-dfdb7bc8d3a1 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:37.414392 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 2/2] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:18:37.422063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766a883-f10e-4041-a171-1470f47ad68c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:37.422608 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766a883-f10e-4041-a171-1470f47ad68c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:37.422608 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766a883-f10e-4041-a171-1470f47ad68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:37.495063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: INFO dbcounter [None req-c766a883-f10e-4041-a171-1470f47ad68c None None] Registered counter for database keystone Aug 29 20:18:37.514886 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG oslo_db.sqlalchemy.engines [None req-c766a883-f10e-4041-a171-1470f47ad68c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81397) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 20:18:37.521788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writer thread running {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 20:18:37.703488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG passlib.handlers.bcrypt [None req-c766a883-f10e-4041-a171-1470f47ad68c None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81397) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 20:18:37.703787 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG passlib.handlers.bcrypt [None req-c766a883-f10e-4041-a171-1470f47ad68c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81397) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 20:18:37.763371 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c766a883-f10e-4041-a171-1470f47ad68c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:37.765298 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c766a883-f10e-4041-a171-1470f47ad68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:37.896479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1/3] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 29 20:18:38.444597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a643a-4c11-4a64-adf8-dfdb7bc8d3a1 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:38.514024 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: INFO dbcounter [None req-1e5a643a-4c11-4a64-adf8-dfdb7bc8d3a1 None None] Registered counter for database keystone Aug 29 20:18:38.536008 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG oslo_db.sqlalchemy.engines [None req-1e5a643a-4c11-4a64-adf8-dfdb7bc8d3a1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81398) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 29 20:18:38.542398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writer thread running {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 29 20:18:38.666090 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a643a-4c11-4a64-adf8-dfdb7bc8d3a1 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:38.670318 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9a39b-6d6d-4ec3-bc96-bfebd23f8bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:38.670947 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9a39b-6d6d-4ec3-bc96-bfebd23f8bad admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:38.671231 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9a39b-6d6d-4ec3-bc96-bfebd23f8bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:38.671508 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9a39b-6d6d-4ec3-bc96-bfebd23f8bad admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:38.679865 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9a39b-6d6d-4ec3-bc96-bfebd23f8bad admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:38.733187 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: WARNING keystone.common.rbac_enforcer.enforcer [None req-88f9a39b-6d6d-4ec3-bc96-bfebd23f8bad admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 20:18:38.733526 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9a39b-6d6d-4ec3-bc96-bfebd23f8bad admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:38.743834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 3/4] 10.222.0.18 () {58 vars in 1324 bytes} [Mon Aug 29 20:18:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:18:38.749846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766a883-f10e-4041-a171-1470f47ad68c None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:38.766117 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766a883-f10e-4041-a171-1470f47ad68c None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:38.771067 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f982d9-6964-4cd4-887c-67ddcc732577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:38.772231 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f982d9-6964-4cd4-887c-67ddcc732577 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:38.772521 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f982d9-6964-4cd4-887c-67ddcc732577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:38.772797 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f982d9-6964-4cd4-887c-67ddcc732577 admin admin] PATH_INFO: `/v3/projects/31796c851dfa4e61a8e1afef74d31165` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:38.774319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f982d9-6964-4cd4-887c-67ddcc732577 admin admin] RBAC: Authorizing `identity:get_project(project_id=31796c851dfa4e61a8e1afef74d31165)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:38.832600 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: WARNING keystone.common.rbac_enforcer.enforcer [None req-07f982d9-6964-4cd4-887c-67ddcc732577 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 29 20:18:38.832970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f982d9-6964-4cd4-887c-67ddcc732577 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:38.838094 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 2/5] 10.222.0.18 () {56 vars in 1207 bytes} [Mon Aug 29 20:18:38 2022] GET /identity/v3/projects/31796c851dfa4e61a8e1afef74d31165 => generated 344 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:18:40.768996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152c39e-13da-41d0-b6fc-ba8394cf3dac None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:40.769489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152c39e-13da-41d0-b6fc-ba8394cf3dac None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:40.769489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152c39e-13da-41d0-b6fc-ba8394cf3dac None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:40.773665 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 4/6] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:18:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:18:40.779289 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76abc9-0cef-4cba-96b0-b1f99173e67b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:40.779698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76abc9-0cef-4cba-96b0-b1f99173e67b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:40.779838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76abc9-0cef-4cba-96b0-b1f99173e67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:40.800859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ad76abc9-0cef-4cba-96b0-b1f99173e67b None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:40.802390 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ad76abc9-0cef-4cba-96b0-b1f99173e67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:40.810282 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 3/7] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 29 20:18:41.079698 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152c39e-13da-41d0-b6fc-ba8394cf3dac None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:41.092235 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152c39e-13da-41d0-b6fc-ba8394cf3dac None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:41.097198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b220f-aeb7-48a0-9ae1-38a8b9d751b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:41.097802 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b220f-aeb7-48a0-9ae1-38a8b9d751b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:41.098084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b220f-aeb7-48a0-9ae1-38a8b9d751b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:41.098359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b220f-aeb7-48a0-9ae1-38a8b9d751b2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:41.106042 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b220f-aeb7-48a0-9ae1-38a8b9d751b2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:41.107805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b220f-aeb7-48a0-9ae1-38a8b9d751b2 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:41.116562 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 5/8] 10.222.0.18 () {58 vars in 1324 bytes} [Mon Aug 29 20:18:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:18:41.121944 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76abc9-0cef-4cba-96b0-b1f99173e67b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:41.131080 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76abc9-0cef-4cba-96b0-b1f99173e67b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:41.135380 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320e436-23f9-4620-999b-f1f197f6c827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:41.136034 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320e436-23f9-4620-999b-f1f197f6c827 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:41.136313 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320e436-23f9-4620-999b-f1f197f6c827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:41.136586 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320e436-23f9-4620-999b-f1f197f6c827 admin admin] PATH_INFO: `/v3/users/b4cb9e8f0d014047a2ac687139b6261f` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:41.137884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320e436-23f9-4620-999b-f1f197f6c827 admin admin] RBAC: Authorizing `identity:get_user(user_id=b4cb9e8f0d014047a2ac687139b6261f)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:41.139698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320e436-23f9-4620-999b-f1f197f6c827 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:41.141251 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 4/9] 10.222.0.18 () {56 vars in 1198 bytes} [Mon Aug 29 20:18:41 2022] GET /identity/v3/users/b4cb9e8f0d014047a2ac687139b6261f => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:18:43.105926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0361d2f-97fe-452c-b6d1-738ff02c63dd None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:43.106633 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0361d2f-97fe-452c-b6d1-738ff02c63dd None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:43.106633 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0361d2f-97fe-452c-b6d1-738ff02c63dd None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:43.107897 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 6/10] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:18:43.113072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcfbd0-fa56-49b3-b0d5-39d19994f1dd None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:43.113491 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcfbd0-fa56-49b3-b0d5-39d19994f1dd None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:43.113631 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcfbd0-fa56-49b3-b0d5-39d19994f1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:43.131970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-63fcfbd0-fa56-49b3-b0d5-39d19994f1dd None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:43.133191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-63fcfbd0-fa56-49b3-b0d5-39d19994f1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:43.179864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46f7c5-6287-44e2-8b53-de8316f41fe8 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:43.180195 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46f7c5-6287-44e2-8b53-de8316f41fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:43.180475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46f7c5-6287-44e2-8b53-de8316f41fe8 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:43.181791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 7/11] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:18:43.186905 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a0d5b-e053-4acc-a8e9-f11e6bc15807 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:43.187192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a0d5b-e053-4acc-a8e9-f11e6bc15807 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:43.187465 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a0d5b-e053-4acc-a8e9-f11e6bc15807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:43.215335 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 5/12] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:18:43.231020 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG passlib.handlers.bcrypt [None req-9e4a0d5b-e053-4acc-a8e9-f11e6bc15807 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81398) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 29 20:18:43.231413 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG passlib.handlers.bcrypt [None req-9e4a0d5b-e053-4acc-a8e9-f11e6bc15807 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81398) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 29 20:18:43.281546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-9e4a0d5b-e053-4acc-a8e9-f11e6bc15807 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:43.283141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9e4a0d5b-e053-4acc-a8e9-f11e6bc15807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:43.386766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 8/13] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:18:43.487244 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fcfbd0-fa56-49b3-b0d5-39d19994f1dd None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:43.511959 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:43.615234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fcfbd0-fa56-49b3-b0d5-39d19994f1dd None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:43.649682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69edd5bb-d982-4be3-bcc8-05e089341092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:43.650242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69edd5bb-d982-4be3-bcc8-05e089341092 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:43.650393 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a0d5b-e053-4acc-a8e9-f11e6bc15807 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:43.650505 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69edd5bb-d982-4be3-bcc8-05e089341092 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:43.650784 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69edd5bb-d982-4be3-bcc8-05e089341092 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:43.667017 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69edd5bb-d982-4be3-bcc8-05e089341092 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:18:43.676522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-69edd5bb-d982-4be3-bcc8-05e089341092 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:18:43.676522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:18:43.676522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:18:43.676522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:18:43.676522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:18:43.676522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:18:43.676522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:18:43.676522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:18:43.676522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:18:43.676522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:18:43.676522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:18:43.676522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:18:43.676522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:18:43.676522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:18:43.676522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:18:43.676522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:18:43.677448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:18:43.677448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:18:43.677448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:18:43.677448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:18:43.677448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:18:43.677448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:18:43.677448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:18:43.677448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:18:43.677448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:18:43.677448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:18:43.677448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:18:43.677448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:18:43.677448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:18:43.677448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:18:43.677448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:18:43.677448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:18:43.678275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:18:43.678275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:18:43.678275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:18:43.678275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:18:43.678275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:18:43.678275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:18:43.678275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:18:43.678275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:18:43.678275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:18:43.678275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:18:43.678275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:18:43.678275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:18:43.678275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 6/14] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:18:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:18:43.683523 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69edd5bb-d982-4be3-bcc8-05e089341092 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:43.760321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a0d5b-e053-4acc-a8e9-f11e6bc15807 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:43.762698 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:43.778472 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:43.798452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69edd5bb-d982-4be3-bcc8-05e089341092 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:43.803262 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d86c0d-a712-4856-aa24-020444bc013e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:43.804160 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d86c0d-a712-4856-aa24-020444bc013e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:43.804447 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d86c0d-a712-4856-aa24-020444bc013e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:43.804720 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d86c0d-a712-4856-aa24-020444bc013e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:43.832471 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74bf1f-f782-4e5e-a5a9-64324fda3abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:43.833084 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74bf1f-f782-4e5e-a5a9-64324fda3abe None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:43.833359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74bf1f-f782-4e5e-a5a9-64324fda3abe None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:43.833628 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74bf1f-f782-4e5e-a5a9-64324fda3abe None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:43.834301 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b74bf1f-f782-4e5e-a5a9-64324fda3abe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:18:43.834599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b74bf1f-f782-4e5e-a5a9-64324fda3abe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:43.835997 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b74bf1f-f782-4e5e-a5a9-64324fda3abe None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:43.848908 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 7/15] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:18:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:18:43.854238 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74bf1f-f782-4e5e-a5a9-64324fda3abe None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:43.877333 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d86c0d-a712-4856-aa24-020444bc013e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:43.878855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d86c0d-a712-4856-aa24-020444bc013e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:43.956568 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b74bf1f-f782-4e5e-a5a9-64324fda3abe None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:43.978529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 9/16] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:18:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 20:18:43.984121 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d86c0d-a712-4856-aa24-020444bc013e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:43.987448 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:43.994285 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700236cc-ffb9-407a-94e3-24c6f1226aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:43.994961 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700236cc-ffb9-407a-94e3-24c6f1226aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:43.995245 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700236cc-ffb9-407a-94e3-24c6f1226aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:43.995557 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700236cc-ffb9-407a-94e3-24c6f1226aa8 None admin] PATH_INFO: `/v3/users/b4cb9e8f0d014047a2ac687139b6261f` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:43.998169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700236cc-ffb9-407a-94e3-24c6f1226aa8 None admin] RBAC: Authorizing `identity:get_user(user_id=b4cb9e8f0d014047a2ac687139b6261f)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:43.999636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700236cc-ffb9-407a-94e3-24c6f1226aa8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:44.001321 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 8/17] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:18:43 2022] GET /identity/v3/users/b4cb9e8f0d014047a2ac687139b6261f => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:18:44.006774 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700236cc-ffb9-407a-94e3-24c6f1226aa8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:44.023583 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:44.102999 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d86c0d-a712-4856-aa24-020444bc013e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:44.124264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700236cc-ffb9-407a-94e3-24c6f1226aa8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:44.136552 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c7d0b-2044-43b2-b509-8d19e82843f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:44.137468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c7d0b-2044-43b2-b509-8d19e82843f1 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:44.137755 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c7d0b-2044-43b2-b509-8d19e82843f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:44.138042 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c7d0b-2044-43b2-b509-8d19e82843f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:44.159693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276436ca-2eaa-4b43-800f-1dc8518080e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:44.159901 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c7d0b-2044-43b2-b509-8d19e82843f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:44.160358 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276436ca-2eaa-4b43-800f-1dc8518080e3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:44.160636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276436ca-2eaa-4b43-800f-1dc8518080e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:44.160903 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276436ca-2eaa-4b43-800f-1dc8518080e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:44.161415 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c7d0b-2044-43b2-b509-8d19e82843f1 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:44.162036 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276436ca-2eaa-4b43-800f-1dc8518080e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:44.163489 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276436ca-2eaa-4b43-800f-1dc8518080e3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:44.165080 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 9/18] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:18:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:18:44.170944 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276436ca-2eaa-4b43-800f-1dc8518080e3 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:44.180470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-420c7d0b-2044-43b2-b509-8d19e82843f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:18:44.180470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:18:44.180470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:18:44.180470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:18:44.180470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:18:44.180470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:18:44.180470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:18:44.180470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:18:44.180470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:18:44.180470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:18:44.180470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:18:44.180470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:18:44.180470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:18:44.180470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:18:44.180470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:18:44.180470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:18:44.181481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:18:44.181481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:18:44.181481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:18:44.181481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:18:44.181481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:18:44.181481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:18:44.181481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:18:44.181481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:18:44.181481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:18:44.181481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:18:44.181481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:18:44.181481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:18:44.181481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:18:44.181481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:18:44.181481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:18:44.181481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:18:44.181481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:18:44.182267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:18:44.182267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:18:44.182267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:18:44.182267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 10/19] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:18:44.187322 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c7d0b-2044-43b2-b509-8d19e82843f1 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:44.244649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:44.252686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:44.280193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276436ca-2eaa-4b43-800f-1dc8518080e3 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:44.295485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c7d0b-2044-43b2-b509-8d19e82843f1 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:44.316822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd3086-2579-497e-aad7-5a583ca5cb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:44.317430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd3086-2579-497e-aad7-5a583ca5cb5c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:44.317706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd3086-2579-497e-aad7-5a583ca5cb5c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:44.317970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd3086-2579-497e-aad7-5a583ca5cb5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:44.318694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd3086-2579-497e-aad7-5a583ca5cb5c None admin] RBAC: Adding query filter params (role.id=322da7bd64024770a64bdb73e5c54719, scope.domain.id=default, user.id=b4cb9e8f0d014047a2ac687139b6261f) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:18:44.318994 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd3086-2579-497e-aad7-5a583ca5cb5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:44.320521 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd3086-2579-497e-aad7-5a583ca5cb5c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:44.326965 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 10/20] 10.222.0.18 () {56 vars in 1322 bytes} [Mon Aug 29 20:18:44 2022] GET /identity/v3/role_assignments?user.id=b4cb9e8f0d014047a2ac687139b6261f&scope.domain.id=default&role.id=322da7bd64024770a64bdb73e5c54719 => generated 233 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:18:44.328632 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053a309-a8ed-4792-a945-b90639631987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:44.329257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053a309-a8ed-4792-a945-b90639631987 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:44.329544 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053a309-a8ed-4792-a945-b90639631987 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:44.329822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053a309-a8ed-4792-a945-b90639631987 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:44.331017 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053a309-a8ed-4792-a945-b90639631987 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:18:44.331323 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053a309-a8ed-4792-a945-b90639631987 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:44.332797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053a309-a8ed-4792-a945-b90639631987 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:44.356217 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 11/21] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:18:46.466762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc1c6a-cf3f-4c8b-9615-072493f0a8c1 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:46.467272 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc1c6a-cf3f-4c8b-9615-072493f0a8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:46.467559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc1c6a-cf3f-4c8b-9615-072493f0a8c1 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:46.469461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 11/22] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:18:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:18:46.474768 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114f6d0-8e0b-42ea-9747-d12cee6158e3 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:46.475618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114f6d0-8e0b-42ea-9747-d12cee6158e3 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:46.476132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114f6d0-8e0b-42ea-9747-d12cee6158e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:46.498545 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2114f6d0-8e0b-42ea-9747-d12cee6158e3 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:46.500658 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2114f6d0-8e0b-42ea-9747-d12cee6158e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:46.585639 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 12/23] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:18:46.662109 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19f85c-32df-420c-90a0-c8b645e3feed None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:46.662741 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19f85c-32df-420c-90a0-c8b645e3feed None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:46.663246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19f85c-32df-420c-90a0-c8b645e3feed None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:46.665044 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 12/24] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:18:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:18:46.670646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151bddb9-1d7c-4601-a710-cf3ab9456de3 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:46.671172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151bddb9-1d7c-4601-a710-cf3ab9456de3 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:46.671842 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151bddb9-1d7c-4601-a710-cf3ab9456de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:46.697015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-151bddb9-1d7c-4601-a710-cf3ab9456de3 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:46.698236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-151bddb9-1d7c-4601-a710-cf3ab9456de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:46.753992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:46.785966 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 13/25] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:18:47.011362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da034f-bd3a-4f0f-bfe1-0fbd2cc04e8c None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:47.012025 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da034f-bd3a-4f0f-bfe1-0fbd2cc04e8c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:47.012509 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da034f-bd3a-4f0f-bfe1-0fbd2cc04e8c None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:47.014092 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 13/26] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:18:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:18:47.019795 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed266a07-6197-415a-810f-73e75c17f7b5 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:47.020466 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed266a07-6197-415a-810f-73e75c17f7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:47.020938 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed266a07-6197-415a-810f-73e75c17f7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:47.023660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da034f-bd3a-4f0f-bfe1-0fbd2cc04e8c None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:47.069542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ed266a07-6197-415a-810f-73e75c17f7b5 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:47.073517 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ed266a07-6197-415a-810f-73e75c17f7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:47.191359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da034f-bd3a-4f0f-bfe1-0fbd2cc04e8c None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:47.195662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 14/27] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:18:47.225490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6fa84-d6b3-4ffc-a2a5-c7aba6037f7c None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:47.226072 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6fa84-d6b3-4ffc-a2a5-c7aba6037f7c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:47.226645 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6fa84-d6b3-4ffc-a2a5-c7aba6037f7c None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:47.228232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 15/28] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:18:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:18:47.232803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e9bb8-f2e0-4eb4-aa4c-c0f16f3292b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:47.233577 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e9bb8-f2e0-4eb4-aa4c-c0f16f3292b5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:47.233942 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e9bb8-f2e0-4eb4-aa4c-c0f16f3292b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:47.234318 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e9bb8-f2e0-4eb4-aa4c-c0f16f3292b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:47.234997 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ba9f4-0fe4-49d2-8380-b020436bc21c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:47.235552 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ba9f4-0fe4-49d2-8380-b020436bc21c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:47.236611 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ba9f4-0fe4-49d2-8380-b020436bc21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:47.250872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e9bb8-f2e0-4eb4-aa4c-c0f16f3292b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:47.260862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e9bb8-f2e0-4eb4-aa4c-c0f16f3292b5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:47.272299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-499ba9f4-0fe4-49d2-8380-b020436bc21c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:47.273657 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-499ba9f4-0fe4-49d2-8380-b020436bc21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:47.292399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:47.296279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-c82e9bb8-f2e0-4eb4-aa4c-c0f16f3292b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:18:47.296279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:18:47.296279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:18:47.296279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:18:47.296279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:18:47.296279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:18:47.296279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:18:47.296279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:18:47.296279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:18:47.296279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:18:47.296279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:18:47.296279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:18:47.296279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:18:47.296279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:18:47.296279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:18:47.296279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:18:47.297124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:18:47.297124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:18:47.297124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:18:47.297124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:18:47.297124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:18:47.297124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:18:47.297124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:18:47.297124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:18:47.297124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:18:47.297124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:18:47.297124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:18:47.297124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:18:47.297124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:18:47.297124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:18:47.297124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:18:47.297124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:18:47.297124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:18:47.298698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:18:47.298698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:18:47.298698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:18:47.310690 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 14/29] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:18:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:18:47.311854 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e9bb8-f2e0-4eb4-aa4c-c0f16f3292b5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:47.378419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 16/30] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:18:47.378419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ba9f4-0fe4-49d2-8380-b020436bc21c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:47.460035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:47.509912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e9bb8-f2e0-4eb4-aa4c-c0f16f3292b5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:47.553616 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7ac6f-c12b-46dd-b099-3d1cd8dcd18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:47.554505 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7ac6f-c12b-46dd-b099-3d1cd8dcd18f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:47.554932 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7ac6f-c12b-46dd-b099-3d1cd8dcd18f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:47.555401 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7ac6f-c12b-46dd-b099-3d1cd8dcd18f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:47.569271 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7ac6f-c12b-46dd-b099-3d1cd8dcd18f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:18:47.575336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-5bf7ac6f-c12b-46dd-b099-3d1cd8dcd18f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:18:47.575336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:18:47.575336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:18:47.575336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:18:47.575336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:18:47.575336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:18:47.575336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:18:47.575336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:18:47.575336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:18:47.575336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:18:47.575336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:18:47.575336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:18:47.575336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:18:47.575336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:18:47.575336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:18:47.575336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:18:47.576220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:18:47.576220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:18:47.576220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:18:47.576220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:18:47.576220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:18:47.576220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:18:47.576220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:18:47.576220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:18:47.576220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:18:47.576220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:18:47.576220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:18:47.576220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:18:47.576220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:18:47.576220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:18:47.576220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:18:47.576220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:18:47.577052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:18:47.577052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:18:47.577052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:18:47.577052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:18:47.577052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:18:47.577052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:18:47.577052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:18:47.577052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:18:47.577052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:18:47.577052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:18:47.577052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:18:47.577052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:18:47.580424 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 15/31] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:18:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:18:47.581486 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7ac6f-c12b-46dd-b099-3d1cd8dcd18f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:47.618137 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ba9f4-0fe4-49d2-8380-b020436bc21c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:47.650161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:47.696192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cc1b2-e461-45bd-8664-2998a4c0271e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:47.698356 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cc1b2-e461-45bd-8664-2998a4c0271e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:47.699793 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cc1b2-e461-45bd-8664-2998a4c0271e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:47.700281 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cc1b2-e461-45bd-8664-2998a4c0271e None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:47.701166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973cc1b2-e461-45bd-8664-2998a4c0271e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:18:47.701683 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973cc1b2-e461-45bd-8664-2998a4c0271e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:47.703879 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973cc1b2-e461-45bd-8664-2998a4c0271e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:47.717899 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 17/32] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:18:47.719094 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cc1b2-e461-45bd-8664-2998a4c0271e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:47.731872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7ac6f-c12b-46dd-b099-3d1cd8dcd18f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:47.800946 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff09bf-cadb-4ea2-b689-1f614f4535f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:47.801836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff09bf-cadb-4ea2-b689-1f614f4535f9 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:47.802232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff09bf-cadb-4ea2-b689-1f614f4535f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:47.802513 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff09bf-cadb-4ea2-b689-1f614f4535f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:47.803858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff09bf-cadb-4ea2-b689-1f614f4535f9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:47.804625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff09bf-cadb-4ea2-b689-1f614f4535f9 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:47.832732 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:47.840094 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cc1b2-e461-45bd-8664-2998a4c0271e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:47.879758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2a21e-d3ef-4eb7-bd58-e56013cd31d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:47.880977 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2a21e-d3ef-4eb7-bd58-e56013cd31d0 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:47.881519 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2a21e-d3ef-4eb7-bd58-e56013cd31d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:47.882111 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2a21e-d3ef-4eb7-bd58-e56013cd31d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:47.883048 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2a21e-d3ef-4eb7-bd58-e56013cd31d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:18:47.883556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2a21e-d3ef-4eb7-bd58-e56013cd31d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:47.885426 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2a21e-d3ef-4eb7-bd58-e56013cd31d0 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:47.899769 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 18/33] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:18:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:18:47.901188 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2a21e-d3ef-4eb7-bd58-e56013cd31d0 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:47.971975 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 16/34] 10.222.0.18 () {60 vars in 1131 bytes} [Mon Aug 29 20:18:47 2022] POST /identity/v3/roles => generated 219 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:18:47.971975 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff09bf-cadb-4ea2-b689-1f614f4535f9 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:48.066353 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2a21e-d3ef-4eb7-bd58-e56013cd31d0 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:48.106675 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbcf03-63d2-4583-a6be-3bc154137f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:48.107423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbcf03-63d2-4583-a6be-3bc154137f61 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:48.107713 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbcf03-63d2-4583-a6be-3bc154137f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:48.108019 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbcf03-63d2-4583-a6be-3bc154137f61 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:48.108947 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbcf03-63d2-4583-a6be-3bc154137f61 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:48.110644 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbcf03-63d2-4583-a6be-3bc154137f61 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:48.136080 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff09bf-cadb-4ea2-b689-1f614f4535f9 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:48.155844 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:48.176746 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8811cfdc-b979-471a-bb56-bde5558c7157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:48.177842 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811cfdc-b979-471a-bb56-bde5558c7157 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:48.178478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811cfdc-b979-471a-bb56-bde5558c7157 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:48.179231 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8811cfdc-b979-471a-bb56-bde5558c7157 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:48.179906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811cfdc-b979-471a-bb56-bde5558c7157 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:48.181510 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8811cfdc-b979-471a-bb56-bde5558c7157 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:48.200730 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 17/35] 10.222.0.18 () {60 vars in 1131 bytes} [Mon Aug 29 20:18:47 2022] POST /identity/v3/roles => generated 225 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:18:48.201639 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8811cfdc-b979-471a-bb56-bde5558c7157 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:48.265736 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 19/36] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 29 20:18:47 2022] POST /identity/v3/projects => generated 301 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:18:48.269246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d8e75-42dd-4d88-96e2-ff1297cf5988 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:48.270004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d8e75-42dd-4d88-96e2-ff1297cf5988 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:48.270500 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d8e75-42dd-4d88-96e2-ff1297cf5988 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:48.272751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 20/37] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:18:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:18:48.279354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77f1e-f782-4c05-a368-d1073128a838 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:48.280118 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77f1e-f782-4c05-a368-d1073128a838 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:48.280600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77f1e-f782-4c05-a368-d1073128a838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:48.303594 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b0a77f1e-f782-4c05-a368-d1073128a838 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:48.306253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b0a77f1e-f782-4c05-a368-d1073128a838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:48.311241 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:48.350149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8811cfdc-b979-471a-bb56-bde5558c7157 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:48.407957 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5156460-9378-4348-b6c4-6a809fac551c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:48.409077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5156460-9378-4348-b6c4-6a809fac551c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:48.409326 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 21/38] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:18:48.409782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5156460-9378-4348-b6c4-6a809fac551c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:48.410267 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5156460-9378-4348-b6c4-6a809fac551c None admin] PATH_INFO: `/v3/users/b4cb9e8f0d014047a2ac687139b6261f` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:48.411712 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5156460-9378-4348-b6c4-6a809fac551c None admin] RBAC: Authorizing `identity:get_user(user_id=b4cb9e8f0d014047a2ac687139b6261f)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:48.416060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5156460-9378-4348-b6c4-6a809fac551c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:48.416756 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 18/39] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:18:48 2022] GET /identity/v3/users/b4cb9e8f0d014047a2ac687139b6261f => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:18:48.422991 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a77f1e-f782-4c05-a368-d1073128a838 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:48.540382 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a77f1e-f782-4c05-a368-d1073128a838 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:48.559021 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced163ab-23ea-43ae-b0b6-883ce00dfa09 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:48.559503 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced163ab-23ea-43ae-b0b6-883ce00dfa09 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:48.559572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced163ab-23ea-43ae-b0b6-883ce00dfa09 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:48.560965 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 19/40] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:18:48.566447 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27ef8c-d610-46de-8381-32b3d116614c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:48.566739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27ef8c-d610-46de-8381-32b3d116614c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:48.567020 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27ef8c-d610-46de-8381-32b3d116614c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:48.577630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32eb67-2db1-4877-a0a5-42023fe5c44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:48.578520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32eb67-2db1-4877-a0a5-42023fe5c44f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:48.578953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32eb67-2db1-4877-a0a5-42023fe5c44f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:48.579915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32eb67-2db1-4877-a0a5-42023fe5c44f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:48.581243 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32eb67-2db1-4877-a0a5-42023fe5c44f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:48.582955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32eb67-2db1-4877-a0a5-42023fe5c44f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:48.588527 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 22/41] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:18:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:18:48.590433 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-7f27ef8c-d610-46de-8381-32b3d116614c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:48.590874 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32eb67-2db1-4877-a0a5-42023fe5c44f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:48.592015 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7f27ef8c-d610-46de-8381-32b3d116614c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:48.606107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:48.643889 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:48.673885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 20/42] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:18:48.701560 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27ef8c-d610-46de-8381-32b3d116614c None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:48.711199 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32eb67-2db1-4877-a0a5-42023fe5c44f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:48.749363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba386f-0d23-43ba-bc13-94578b1a15de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:48.750146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba386f-0d23-43ba-bc13-94578b1a15de None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:48.750429 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba386f-0d23-43ba-bc13-94578b1a15de None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:48.750709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba386f-0d23-43ba-bc13-94578b1a15de None admin] PATH_INFO: `/v3/domains/default/users/b4cb9e8f0d014047a2ac687139b6261f/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:48.753668 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba386f-0d23-43ba-bc13-94578b1a15de None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b4cb9e8f0d014047a2ac687139b6261f, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:48.755191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba386f-0d23-43ba-bc13-94578b1a15de None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:48.771705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 23/43] 10.222.0.18 () {58 vars in 1361 bytes} [Mon Aug 29 20:18:48 2022] PUT /identity/v3/domains/default/users/b4cb9e8f0d014047a2ac687139b6261f/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:18:48.808776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27ef8c-d610-46de-8381-32b3d116614c None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:48.847372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a2e90-0ade-4fd2-9cda-59af2932a724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:48.848041 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a2e90-0ade-4fd2-9cda-59af2932a724 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:48.848323 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a2e90-0ade-4fd2-9cda-59af2932a724 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:48.848592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a2e90-0ade-4fd2-9cda-59af2932a724 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:48.849272 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a2e90-0ade-4fd2-9cda-59af2932a724 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:48.850655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a2e90-0ade-4fd2-9cda-59af2932a724 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:48.867256 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 21/44] 10.222.0.18 () {60 vars in 1131 bytes} [Mon Aug 29 20:18:48 2022] POST /identity/v3/roles => generated 223 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:18:48.949845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba386f-0d23-43ba-bc13-94578b1a15de None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:49.023977 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:49.061922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba386f-0d23-43ba-bc13-94578b1a15de None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:49.098553 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240b7ca-40cf-4565-bffe-1d72ebb65353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:49.099198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240b7ca-40cf-4565-bffe-1d72ebb65353 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:49.099481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240b7ca-40cf-4565-bffe-1d72ebb65353 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:49.099758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240b7ca-40cf-4565-bffe-1d72ebb65353 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:49.100875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240b7ca-40cf-4565-bffe-1d72ebb65353 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:49.102317 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240b7ca-40cf-4565-bffe-1d72ebb65353 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:49.104095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 24/45] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:18:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:18:49.109433 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a2e90-0ade-4fd2-9cda-59af2932a724 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:49.145498 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:49.212376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a2e90-0ade-4fd2-9cda-59af2932a724 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:49.247172 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c32fef-54de-459d-89ee-c00a350a8175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:49.247790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c32fef-54de-459d-89ee-c00a350a8175 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:49.248102 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c32fef-54de-459d-89ee-c00a350a8175 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:49.248382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c32fef-54de-459d-89ee-c00a350a8175 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:49.249032 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c32fef-54de-459d-89ee-c00a350a8175 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:49.250499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c32fef-54de-459d-89ee-c00a350a8175 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:49.280619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 22/46] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 29 20:18:49 2022] POST /identity/v3/projects => generated 312 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:18:50.705686 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0f235-2e4f-4a72-9e9c-2ba4a2467f06 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:50.706193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0f235-2e4f-4a72-9e9c-2ba4a2467f06 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:50.706193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0f235-2e4f-4a72-9e9c-2ba4a2467f06 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:50.707504 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 25/47] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:18:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:18:50.712902 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab88c7-39e7-4df6-a073-39c06a5fc60c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:50.713304 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab88c7-39e7-4df6-a073-39c06a5fc60c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:50.713488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab88c7-39e7-4df6-a073-39c06a5fc60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:50.732193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1fab88c7-39e7-4df6-a073-39c06a5fc60c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:50.733391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1fab88c7-39e7-4df6-a073-39c06a5fc60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:50.907602 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 23/48] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:18:51.178078 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0f235-2e4f-4a72-9e9c-2ba4a2467f06 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:51.240942 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b748db9-24f9-4119-8177-b9ab67ee09cd None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:51.241379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b748db9-24f9-4119-8177-b9ab67ee09cd None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:51.241665 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b748db9-24f9-4119-8177-b9ab67ee09cd None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:51.243033 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 24/49] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:18:51.248249 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5969df9-b80e-498d-a4db-603e762891da None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:51.248530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5969df9-b80e-498d-a4db-603e762891da None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:51.248805 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5969df9-b80e-498d-a4db-603e762891da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:51.266021 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b5969df9-b80e-498d-a4db-603e762891da None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:51.267202 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b5969df9-b80e-498d-a4db-603e762891da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:51.278462 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:51.283373 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0f235-2e4f-4a72-9e9c-2ba4a2467f06 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:51.317174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3789e-e7fb-4674-bff6-17775d1e5058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:51.317818 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3789e-e7fb-4674-bff6-17775d1e5058 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:51.318106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3789e-e7fb-4674-bff6-17775d1e5058 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:51.318387 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3789e-e7fb-4674-bff6-17775d1e5058 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:51.321941 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:51.330764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3789e-e7fb-4674-bff6-17775d1e5058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:18:51.336875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-d7e3789e-e7fb-4674-bff6-17775d1e5058 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:18:51.336875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:18:51.336875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:18:51.336875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:18:51.336875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:18:51.336875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:18:51.336875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:18:51.336875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:18:51.336875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:18:51.336875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:18:51.336875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:18:51.336875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:18:51.336875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:18:51.336875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:18:51.336875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:18:51.336875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:18:51.337701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:18:51.337701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:18:51.337701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:18:51.337701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:18:51.337701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:18:51.337701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:18:51.337701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:18:51.337701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:18:51.337701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:18:51.337701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:18:51.337701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:18:51.337701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:18:51.337701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:18:51.337701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:18:51.337701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:18:51.337701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:18:51.338425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:18:51.338425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:18:51.338425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:18:51.338425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:18:51.338425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:18:51.338425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:18:51.338425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:18:51.338425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:18:51.338425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:18:51.338425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:18:51.338425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:18:51.338425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:18:51.338425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 26/51] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:18:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:18:51.339087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 25/51] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:18:51.344359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3789e-e7fb-4674-bff6-17775d1e5058 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:51.443392 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3789e-e7fb-4674-bff6-17775d1e5058 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:51.481317 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234110cf-1f7e-45ae-ba19-4f4e9ec3da55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:51.481946 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234110cf-1f7e-45ae-ba19-4f4e9ec3da55 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:51.482233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234110cf-1f7e-45ae-ba19-4f4e9ec3da55 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:51.482514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234110cf-1f7e-45ae-ba19-4f4e9ec3da55 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:51.483191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234110cf-1f7e-45ae-ba19-4f4e9ec3da55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:18:51.483502 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234110cf-1f7e-45ae-ba19-4f4e9ec3da55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:51.484948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234110cf-1f7e-45ae-ba19-4f4e9ec3da55 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:51.494732 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 27/52] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:18:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:18:51.500449 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5969df9-b80e-498d-a4db-603e762891da None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:51.599654 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5969df9-b80e-498d-a4db-603e762891da None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:51.606282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234110cf-1f7e-45ae-ba19-4f4e9ec3da55 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:51.640967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e6222-bb41-4643-9c0e-4dba94d49118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:51.642475 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e6222-bb41-4643-9c0e-4dba94d49118 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:51.642475 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e6222-bb41-4643-9c0e-4dba94d49118 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:51.642853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e6222-bb41-4643-9c0e-4dba94d49118 None admin] PATH_INFO: `/v3/users/b4cb9e8f0d014047a2ac687139b6261f` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:51.644044 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e6222-bb41-4643-9c0e-4dba94d49118 None admin] RBAC: Authorizing `identity:get_user(user_id=b4cb9e8f0d014047a2ac687139b6261f)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:51.645503 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e6222-bb41-4643-9c0e-4dba94d49118 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:51.647126 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 26/53] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:18:51 2022] GET /identity/v3/users/b4cb9e8f0d014047a2ac687139b6261f => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:18:51.652975 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e6222-bb41-4643-9c0e-4dba94d49118 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:51.668914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:51.669655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:51.712003 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234110cf-1f7e-45ae-ba19-4f4e9ec3da55 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:51.759688 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e6222-bb41-4643-9c0e-4dba94d49118 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:51.796960 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadd8fc-408f-42f1-b040-0ead12021af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:51.797606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadd8fc-408f-42f1-b040-0ead12021af1 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:51.798388 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadd8fc-408f-42f1-b040-0ead12021af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:51.798746 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadd8fc-408f-42f1-b040-0ead12021af1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:51.800343 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadd8fc-408f-42f1-b040-0ead12021af1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:51.801893 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadd8fc-408f-42f1-b040-0ead12021af1 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:51.803559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 27/54] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:18:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:18:51.809745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadd8fc-408f-42f1-b040-0ead12021af1 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:51.855473 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8d4fa-5ee4-4a59-87ef-45df684e17be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:51.856146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8d4fa-5ee4-4a59-87ef-45df684e17be None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:51.856434 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8d4fa-5ee4-4a59-87ef-45df684e17be None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:51.856716 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8d4fa-5ee4-4a59-87ef-45df684e17be None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:51.857833 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8d4fa-5ee4-4a59-87ef-45df684e17be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:51.859290 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8d4fa-5ee4-4a59-87ef-45df684e17be None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:51.861697 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 28/55] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:18:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:18:51.867171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8d4fa-5ee4-4a59-87ef-45df684e17be None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:51.891076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:51.911031 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadd8fc-408f-42f1-b040-0ead12021af1 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:51.945688 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952701c-a03a-40a1-a77f-80caeab14eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:51.946302 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952701c-a03a-40a1-a77f-80caeab14eca None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:51.946580 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952701c-a03a-40a1-a77f-80caeab14eca None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:51.946855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952701c-a03a-40a1-a77f-80caeab14eca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:51.947591 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952701c-a03a-40a1-a77f-80caeab14eca None admin] RBAC: Adding query filter params (role.id=322da7bd64024770a64bdb73e5c54719, scope.domain.id=default, user.id=b4cb9e8f0d014047a2ac687139b6261f) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:18:51.947901 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952701c-a03a-40a1-a77f-80caeab14eca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:51.949450 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952701c-a03a-40a1-a77f-80caeab14eca None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:51.955621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 28/56] 10.222.0.18 () {56 vars in 1322 bytes} [Mon Aug 29 20:18:51 2022] GET /identity/v3/role_assignments?user.id=b4cb9e8f0d014047a2ac687139b6261f&scope.domain.id=default&role.id=322da7bd64024770a64bdb73e5c54719 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:18:51.975028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8d4fa-5ee4-4a59-87ef-45df684e17be None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:52.005735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:52.016058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8725e849-0277-4ddd-9c8a-35dcc203f443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:52.016723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725e849-0277-4ddd-9c8a-35dcc203f443 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:52.017013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725e849-0277-4ddd-9c8a-35dcc203f443 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:52.017294 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725e849-0277-4ddd-9c8a-35dcc203f443 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:52.017970 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8725e849-0277-4ddd-9c8a-35dcc203f443 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:52.019482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8725e849-0277-4ddd-9c8a-35dcc203f443 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:52.049763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 29/57] 10.222.0.18 () {60 vars in 1140 bytes} [Mon Aug 29 20:18:51 2022] POST /identity/v3/projects => generated 298 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:18:53.964368 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ba674-90ca-4a92-b0de-4f01b5a8653e None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:53.964883 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ba674-90ca-4a92-b0de-4f01b5a8653e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:53.964968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ba674-90ca-4a92-b0de-4f01b5a8653e None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:53.966291 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 29/58] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:18:53.972054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e88eb3-7459-4c27-a108-59b2c33d0ebd None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:53.972486 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e88eb3-7459-4c27-a108-59b2c33d0ebd None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:53.972624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e88eb3-7459-4c27-a108-59b2c33d0ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:53.995006 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-14e88eb3-7459-4c27-a108-59b2c33d0ebd None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:53.996242 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-14e88eb3-7459-4c27-a108-59b2c33d0ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:54.060626 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 30/59] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:18:54.346998 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ba674-90ca-4a92-b0de-4f01b5a8653e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:54.447949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ba674-90ca-4a92-b0de-4f01b5a8653e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:54.480147 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8c29a-8e4a-410b-aafe-86c4e947531d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:54.480779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8c29a-8e4a-410b-aafe-86c4e947531d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:54.481064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8c29a-8e4a-410b-aafe-86c4e947531d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:54.481336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8c29a-8e4a-410b-aafe-86c4e947531d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:54.482439 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8c29a-8e4a-410b-aafe-86c4e947531d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:54.483875 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f8c29a-8e4a-410b-aafe-86c4e947531d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:54.485471 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 30/60] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:18:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:18:54.491560 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e88eb3-7459-4c27-a108-59b2c33d0ebd None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:54.588935 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e88eb3-7459-4c27-a108-59b2c33d0ebd None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:54.598175 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:54.623711 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b06c769-d67b-4606-8fb8-dc65ce8c2c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:54.624355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06c769-d67b-4606-8fb8-dc65ce8c2c46 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:54.624641 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06c769-d67b-4606-8fb8-dc65ce8c2c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:54.624917 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b06c769-d67b-4606-8fb8-dc65ce8c2c46 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:54.625575 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06c769-d67b-4606-8fb8-dc65ce8c2c46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:54.627671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b06c769-d67b-4606-8fb8-dc65ce8c2c46 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:54.653549 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 31/61] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 29 20:18:54 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:18:57.627465 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103a657-bdf6-4364-b750-d6eb2354445e None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:57.628589 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103a657-bdf6-4364-b750-d6eb2354445e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:57.628589 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103a657-bdf6-4364-b750-d6eb2354445e None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:57.630117 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 31/62] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:18:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:18:57.635815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ddcf6-5f18-4429-a51b-4255a522a883 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:57.636323 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ddcf6-5f18-4429-a51b-4255a522a883 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:57.636394 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ddcf6-5f18-4429-a51b-4255a522a883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:57.665153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-216ddcf6-5f18-4429-a51b-4255a522a883 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:57.666552 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-216ddcf6-5f18-4429-a51b-4255a522a883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:57.767800 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 32/63] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:18:57.935093 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3a71f-5c45-4319-a538-10466e1d8c68 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:57.935413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3a71f-5c45-4319-a538-10466e1d8c68 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:57.935583 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3a71f-5c45-4319-a538-10466e1d8c68 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:57.936964 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 32/64] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:18:57.942352 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149469d-c66c-4015-873b-fbda05096ad0 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:57.942655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149469d-c66c-4015-873b-fbda05096ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:57.942941 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149469d-c66c-4015-873b-fbda05096ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:57.967645 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-c149469d-c66c-4015-873b-fbda05096ad0 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:57.968910 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c149469d-c66c-4015-873b-fbda05096ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:58.022062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece485b0-6157-42d8-bb92-a3e755800c37 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:58.022360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece485b0-6157-42d8-bb92-a3e755800c37 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:58.022621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece485b0-6157-42d8-bb92-a3e755800c37 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:58.024292 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 33/65] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:18:58.031899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b348e-f750-4198-9dc3-40a299d2a3ec None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:58.032219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b348e-f750-4198-9dc3-40a299d2a3ec None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:58.032504 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b348e-f750-4198-9dc3-40a299d2a3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:58.055463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 33/66] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:18:58.064212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-313b348e-f750-4198-9dc3-40a299d2a3ec None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:58.065764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-313b348e-f750-4198-9dc3-40a299d2a3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:58.078663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:58.146003 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab935aa7-07ae-45f0-b845-43740dec136f None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:58.146359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab935aa7-07ae-45f0-b845-43740dec136f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:58.146865 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab935aa7-07ae-45f0-b845-43740dec136f None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:58.148447 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 34/67] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:18:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:18:58.149343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab935aa7-07ae-45f0-b845-43740dec136f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:58.160294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 34/68] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:18:58.161416 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f12eb-f32b-4bb1-bf9b-5c3fce0482de None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:58.161711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f12eb-f32b-4bb1-bf9b-5c3fce0482de None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:58.161988 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f12eb-f32b-4bb1-bf9b-5c3fce0482de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:58.182500 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-f93f12eb-f32b-4bb1-bf9b-5c3fce0482de None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:58.184206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f93f12eb-f32b-4bb1-bf9b-5c3fce0482de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:58.186704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:58.293519 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 35/69] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:18:58.346773 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab935aa7-07ae-45f0-b845-43740dec136f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:58.396332 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207ae22-e0d9-48ba-b523-c16703269c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:58.397049 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207ae22-e0d9-48ba-b523-c16703269c9e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:58.397341 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207ae22-e0d9-48ba-b523-c16703269c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:58.397627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207ae22-e0d9-48ba-b523-c16703269c9e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:58.417291 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207ae22-e0d9-48ba-b523-c16703269c9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:18:58.421426 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cb971-8573-4b67-815e-045a7897b25b None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:58.421718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cb971-8573-4b67-815e-045a7897b25b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:58.421995 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cb971-8573-4b67-815e-045a7897b25b None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:58.423363 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 36/70] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:18:58.426715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-9207ae22-e0d9-48ba-b523-c16703269c9e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:18:58.426715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:18:58.426715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:18:58.426715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:18:58.426715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:18:58.426715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:18:58.426715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:18:58.426715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:18:58.426715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:18:58.426715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:18:58.426715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:18:58.426715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:18:58.426715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:18:58.426715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:18:58.426715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:18:58.426715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:18:58.427564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:18:58.427564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:18:58.427564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:18:58.427564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:18:58.427564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:18:58.427564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:18:58.427564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:18:58.427564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:18:58.427564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:18:58.427564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:18:58.427564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:18:58.427564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:18:58.427564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:18:58.427564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:18:58.427564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:18:58.427564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:18:58.428363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:18:58.428363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:18:58.428363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:18:58.428363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:18:58.428363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:18:58.428363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:18:58.428363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:18:58.428363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:18:58.428363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:18:58.428363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:18:58.428363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:18:58.428363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:18:58.428363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 35/71] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:18:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:18:58.434483 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062cb971-8573-4b67-815e-045a7897b25b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:58.437165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebf2e5-fe79-46fd-b143-516c9890f495 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:58.437402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebf2e5-fe79-46fd-b143-516c9890f495 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:58.437686 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebf2e5-fe79-46fd-b143-516c9890f495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:58.456349 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-73ebf2e5-fe79-46fd-b143-516c9890f495 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:18:58.457643 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-73ebf2e5-fe79-46fd-b143-516c9890f495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:18:58.519120 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:58.528819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 36/72] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:18:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:18:58.532852 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebf2e5-fe79-46fd-b143-516c9890f495 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:58.550446 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:58.554367 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062cb971-8573-4b67-815e-045a7897b25b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:58.590634 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1f82d-8eb3-418b-bd4d-0b1dc48060ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:58.591590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1f82d-8eb3-418b-bd4d-0b1dc48060ed None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:58.592151 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1f82d-8eb3-418b-bd4d-0b1dc48060ed None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:58.592617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1f82d-8eb3-418b-bd4d-0b1dc48060ed None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:58.593512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1f82d-8eb3-418b-bd4d-0b1dc48060ed None admin] RBAC: Adding query filter params (name=member) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:18:58.594018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1f82d-8eb3-418b-bd4d-0b1dc48060ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:58.595704 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1f82d-8eb3-418b-bd4d-0b1dc48060ed None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:58.604968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 37/73] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:18:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:18:58.606181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1f82d-8eb3-418b-bd4d-0b1dc48060ed None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:58.641319 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebf2e5-fe79-46fd-b143-516c9890f495 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:58.679608 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15d901-d114-4bcb-ac55-b82cbb2bfd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:58.680332 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15d901-d114-4bcb-ac55-b82cbb2bfd35 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:58.680621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15d901-d114-4bcb-ac55-b82cbb2bfd35 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:58.680903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15d901-d114-4bcb-ac55-b82cbb2bfd35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:58.692417 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15d901-d114-4bcb-ac55-b82cbb2bfd35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:18:58.698064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-aa15d901-d114-4bcb-ac55-b82cbb2bfd35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:18:58.698064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:18:58.698064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:18:58.698064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:18:58.698064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:18:58.698064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:18:58.698064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:18:58.698064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:18:58.698064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:18:58.698064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:18:58.698064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:18:58.698064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:18:58.698064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:18:58.698064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:18:58.698064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:18:58.698064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:18:58.698902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:18:58.698902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:18:58.698902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:18:58.698902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:18:58.698902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:18:58.698902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:18:58.698902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:18:58.698902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:18:58.698902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:18:58.698902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:18:58.698902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:18:58.698902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:18:58.698902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:18:58.698902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:18:58.698902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:18:58.698902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:18:58.699706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:18:58.699706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:18:58.699706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:18:58.699706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:18:58.699706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:18:58.699706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:18:58.699706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:18:58.699706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:18:58.699706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:18:58.699706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:18:58.699706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:18:58.699706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:18:58.699706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 37/74] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:18:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:18:58.700357 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15d901-d114-4bcb-ac55-b82cbb2bfd35 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:58.714902 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1f82d-8eb3-418b-bd4d-0b1dc48060ed None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:58.756493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f94c592-f917-45f7-918e-b6a657a8ff46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:58.757140 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94c592-f917-45f7-918e-b6a657a8ff46 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:58.757430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94c592-f917-45f7-918e-b6a657a8ff46 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:58.757701 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94c592-f917-45f7-918e-b6a657a8ff46 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:58.760430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:58.771488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f94c592-f917-45f7-918e-b6a657a8ff46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:18:58.777788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-2f94c592-f917-45f7-918e-b6a657a8ff46 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:18:58.777788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:18:58.777788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:18:58.777788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:18:58.777788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:18:58.777788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:18:58.777788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:18:58.777788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:18:58.777788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:18:58.777788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:18:58.777788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:18:58.777788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:18:58.777788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:18:58.777788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:18:58.777788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:18:58.777788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:18:58.778621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:18:58.778621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:18:58.778621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:18:58.778621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:18:58.778621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:18:58.778621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:18:58.778621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:18:58.778621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:18:58.778621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:18:58.778621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:18:58.778621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:18:58.778621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:18:58.778621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:18:58.778621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:18:58.778621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:18:58.778621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:18:58.779372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:18:58.779372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:18:58.779372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:18:58.779372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:18:58.779372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:18:58.779372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:18:58.779372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:18:58.779372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:18:58.779372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:18:58.779372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:18:58.779372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:18:58.779372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:18:58.779372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 38/75] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:18:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:18:58.779372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f94c592-f917-45f7-918e-b6a657a8ff46 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:58.786365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:58.813213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15d901-d114-4bcb-ac55-b82cbb2bfd35 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:58.847763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f82a73-7d3d-478e-bbb5-a9c6ec1337d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:58.848442 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f82a73-7d3d-478e-bbb5-a9c6ec1337d0 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:58.848727 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f82a73-7d3d-478e-bbb5-a9c6ec1337d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:58.849004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f82a73-7d3d-478e-bbb5-a9c6ec1337d0 None admin] PATH_INFO: `/v3/users/0c802f73a7a64d1b862c3589c9d42f5b` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:58.866022 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f82a73-7d3d-478e-bbb5-a9c6ec1337d0 None admin] RBAC: Authorizing `identity:get_user(user_id=0c802f73a7a64d1b862c3589c9d42f5b)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:58.867632 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f82a73-7d3d-478e-bbb5-a9c6ec1337d0 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:58.869346 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 38/76] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:18:58 2022] GET /identity/v3/users/0c802f73a7a64d1b862c3589c9d42f5b => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:18:58.870133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f82a73-7d3d-478e-bbb5-a9c6ec1337d0 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:58.900736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f94c592-f917-45f7-918e-b6a657a8ff46 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:58.938141 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd3491-45a7-4243-872a-36cf5bda70e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:58.938800 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd3491-45a7-4243-872a-36cf5bda70e9 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:58.939079 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd3491-45a7-4243-872a-36cf5bda70e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:58.939359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd3491-45a7-4243-872a-36cf5bda70e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:58.940524 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd3491-45a7-4243-872a-36cf5bda70e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:58.942604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd3491-45a7-4243-872a-36cf5bda70e9 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:58.944161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 39/77] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:18:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:18:58.945127 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd3491-45a7-4243-872a-36cf5bda70e9 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:58.980612 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f82a73-7d3d-478e-bbb5-a9c6ec1337d0 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:59.011672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:59.012416 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:59.017990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66482dce-727d-4cec-ad79-ad93b1736f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:59.019297 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66482dce-727d-4cec-ad79-ad93b1736f7c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:59.019691 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66482dce-727d-4cec-ad79-ad93b1736f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:59.020003 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66482dce-727d-4cec-ad79-ad93b1736f7c None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:59.020739 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66482dce-727d-4cec-ad79-ad93b1736f7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:18:59.021055 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66482dce-727d-4cec-ad79-ad93b1736f7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:59.022538 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66482dce-727d-4cec-ad79-ad93b1736f7c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:59.032393 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 39/78] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:18:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:18:59.033080 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66482dce-727d-4cec-ad79-ad93b1736f7c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:59.062340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd3491-45a7-4243-872a-36cf5bda70e9 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:59.104689 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4792a57-df27-44a5-801f-93fc4a0677e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:59.105322 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4792a57-df27-44a5-801f-93fc4a0677e8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:59.105691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4792a57-df27-44a5-801f-93fc4a0677e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:59.105974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4792a57-df27-44a5-801f-93fc4a0677e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:59.106658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4792a57-df27-44a5-801f-93fc4a0677e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:18:59.106963 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4792a57-df27-44a5-801f-93fc4a0677e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:59.108401 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4792a57-df27-44a5-801f-93fc4a0677e8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:59.117383 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 40/79] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:18:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:18:59.118298 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4792a57-df27-44a5-801f-93fc4a0677e8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:59.158024 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66482dce-727d-4cec-ad79-ad93b1736f7c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:59.194141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215742a-66de-4d86-8478-2c93d0451bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:59.194707 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215742a-66de-4d86-8478-2c93d0451bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:59.194998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215742a-66de-4d86-8478-2c93d0451bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:59.195275 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215742a-66de-4d86-8478-2c93d0451bc4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:59.206763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215742a-66de-4d86-8478-2c93d0451bc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:18:59.213122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-4215742a-66de-4d86-8478-2c93d0451bc4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:18:59.213122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:18:59.213122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:18:59.213122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:18:59.213122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:18:59.213122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:18:59.213122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:18:59.213122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:18:59.213122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:18:59.213122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:18:59.213122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:18:59.213122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:18:59.213122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:18:59.213122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:18:59.213122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:18:59.213122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:18:59.214014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:18:59.214014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:18:59.214014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:18:59.214014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:18:59.214014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:18:59.214014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:18:59.214014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:18:59.214014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:18:59.214014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:18:59.214014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:18:59.214014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:18:59.214014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:18:59.214014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:18:59.214014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:18:59.214014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:18:59.214014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:18:59.214775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:18:59.214775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:18:59.214775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:18:59.214775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:18:59.214775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:18:59.214775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:18:59.214775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:18:59.214775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:18:59.214775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:18:59.214775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:18:59.214775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:18:59.214775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:18:59.214775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 40/80] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:18:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:18:59.215385 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215742a-66de-4d86-8478-2c93d0451bc4 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:59.243536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4792a57-df27-44a5-801f-93fc4a0677e8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:59.272390 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:59.285341 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714e1ee-e15c-4c78-8745-03aea73e75e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:59.286291 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714e1ee-e15c-4c78-8745-03aea73e75e2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:59.286571 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714e1ee-e15c-4c78-8745-03aea73e75e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:59.286847 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714e1ee-e15c-4c78-8745-03aea73e75e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:59.294809 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:59.350704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215742a-66de-4d86-8478-2c93d0451bc4 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:59.359570 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5714e1ee-e15c-4c78-8745-03aea73e75e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:59.361285 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5714e1ee-e15c-4c78-8745-03aea73e75e2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:59.393858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27aa4a4-b82e-4811-92c6-be6794f37b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:59.394558 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27aa4a4-b82e-4811-92c6-be6794f37b1c None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:59.394846 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27aa4a4-b82e-4811-92c6-be6794f37b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:59.395123 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27aa4a4-b82e-4811-92c6-be6794f37b1c None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:59.395795 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27aa4a4-b82e-4811-92c6-be6794f37b1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:59.397784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27aa4a4-b82e-4811-92c6-be6794f37b1c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:59.421770 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 41/81] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 29 20:18:59 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:18:59.424143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27aa4a4-b82e-4811-92c6-be6794f37b1c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:59.486922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 41/82] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:18:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:18:59.492561 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714e1ee-e15c-4c78-8745-03aea73e75e2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:59.523341 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:59.556416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27aa4a4-b82e-4811-92c6-be6794f37b1c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:59.569834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:59.605697 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177eb0b7-f9ab-407a-ad93-cffa42d84a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:59.606538 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177eb0b7-f9ab-407a-ad93-cffa42d84a84 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:59.606664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177eb0b7-f9ab-407a-ad93-cffa42d84a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:59.607965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177eb0b7-f9ab-407a-ad93-cffa42d84a84 None admin] PATH_INFO: `/v3/users/b4cb9e8f0d014047a2ac687139b6261f` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:59.608117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177eb0b7-f9ab-407a-ad93-cffa42d84a84 None admin] RBAC: Authorizing `identity:get_user(user_id=b4cb9e8f0d014047a2ac687139b6261f)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:59.609600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177eb0b7-f9ab-407a-ad93-cffa42d84a84 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:59.611260 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 42/83] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:18:59 2022] GET /identity/v3/users/b4cb9e8f0d014047a2ac687139b6261f => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:18:59.616083 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177eb0b7-f9ab-407a-ad93-cffa42d84a84 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:59.622526 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714e1ee-e15c-4c78-8745-03aea73e75e2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:59.673229 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b038e4-4e67-4c25-9f91-4356675fb43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:59.674799 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b038e4-4e67-4c25-9f91-4356675fb43c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:59.675292 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b038e4-4e67-4c25-9f91-4356675fb43c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:59.675907 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b038e4-4e67-4c25-9f91-4356675fb43c None admin] PATH_INFO: `/v3/users/0c802f73a7a64d1b862c3589c9d42f5b` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:59.679000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b038e4-4e67-4c25-9f91-4356675fb43c None admin] RBAC: Authorizing `identity:get_user(user_id=0c802f73a7a64d1b862c3589c9d42f5b)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:59.680396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b038e4-4e67-4c25-9f91-4356675fb43c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:59.682051 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 42/84] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:18:59 2022] GET /identity/v3/users/0c802f73a7a64d1b862c3589c9d42f5b => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:18:59.682985 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b038e4-4e67-4c25-9f91-4356675fb43c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:59.739749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177eb0b7-f9ab-407a-ad93-cffa42d84a84 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:59.792067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca2e56-b590-4cb8-ba83-6983227e3b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:59.793014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca2e56-b590-4cb8-ba83-6983227e3b84 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:59.793364 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca2e56-b590-4cb8-ba83-6983227e3b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:59.794147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca2e56-b590-4cb8-ba83-6983227e3b84 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:59.794473 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca2e56-b590-4cb8-ba83-6983227e3b84 None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:18:59.795089 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca2e56-b590-4cb8-ba83-6983227e3b84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:59.796404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca2e56-b590-4cb8-ba83-6983227e3b84 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:59.806822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 43/85] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:18:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:18:59.812435 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca2e56-b590-4cb8-ba83-6983227e3b84 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:59.818520 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:59.828703 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b038e4-4e67-4c25-9f91-4356675fb43c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:18:59.873930 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211adf2-e5d9-4261-8366-f1a546244728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:18:59.875153 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211adf2-e5d9-4261-8366-f1a546244728 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:18:59.875826 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211adf2-e5d9-4261-8366-f1a546244728 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:18:59.876691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211adf2-e5d9-4261-8366-f1a546244728 None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:18:59.879778 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211adf2-e5d9-4261-8366-f1a546244728 None admin] RBAC: Authorizing `identity:get_project(project_id=0370c388ce4e42d2ad10ebca467e24dc)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:18:59.881497 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211adf2-e5d9-4261-8366-f1a546244728 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:18:59.884236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 43/86] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:18:59 2022] GET /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:18:59.885191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211adf2-e5d9-4261-8366-f1a546244728 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:18:59.892491 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:18:59.963424 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca2e56-b590-4cb8-ba83-6983227e3b84 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:00.015615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11952af0-9bb4-4546-a246-831f2496f131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:00.016209 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11952af0-9bb4-4546-a246-831f2496f131 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:00.016546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11952af0-9bb4-4546-a246-831f2496f131 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:00.016984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11952af0-9bb4-4546-a246-831f2496f131 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:00.026209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211adf2-e5d9-4261-8366-f1a546244728 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:00.074529 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614d2cb-aa61-43c4-a734-766dc2322b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:00.075180 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614d2cb-aa61-43c4-a734-766dc2322b3a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:00.075716 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614d2cb-aa61-43c4-a734-766dc2322b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:00.076033 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614d2cb-aa61-43c4-a734-766dc2322b3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:00.110206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11952af0-9bb4-4546-a246-831f2496f131 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:00.112083 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11952af0-9bb4-4546-a246-831f2496f131 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:00.121573 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:00.170400 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:00.182294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614d2cb-aa61-43c4-a734-766dc2322b3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:00.184138 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614d2cb-aa61-43c4-a734-766dc2322b3a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:00.254794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 44/87] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:18:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:00.255886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11952af0-9bb4-4546-a246-831f2496f131 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:00.316176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 44/88] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:18:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:00.317128 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614d2cb-aa61-43c4-a734-766dc2322b3a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:00.374395 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11952af0-9bb4-4546-a246-831f2496f131 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:00.386193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:00.419152 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bab51-befe-4526-9ad0-d0de45be4ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:00.420266 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bab51-befe-4526-9ad0-d0de45be4ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:00.420767 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bab51-befe-4526-9ad0-d0de45be4ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:00.421305 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bab51-befe-4526-9ad0-d0de45be4ed4 None admin] PATH_INFO: `/v3/users/0c802f73a7a64d1b862c3589c9d42f5b` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:00.422884 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888bab51-befe-4526-9ad0-d0de45be4ed4 None admin] RBAC: Authorizing `identity:get_user(user_id=0c802f73a7a64d1b862c3589c9d42f5b)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:00.425030 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888bab51-befe-4526-9ad0-d0de45be4ed4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:00.427757 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 45/89] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:00 2022] GET /identity/v3/users/0c802f73a7a64d1b862c3589c9d42f5b => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:00.428927 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bab51-befe-4526-9ad0-d0de45be4ed4 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:00.443482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:00.445008 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614d2cb-aa61-43c4-a734-766dc2322b3a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:00.483791 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb4487-899a-4c3b-b648-557870c7e756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:00.484467 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb4487-899a-4c3b-b648-557870c7e756 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:00.484751 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb4487-899a-4c3b-b648-557870c7e756 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:00.485029 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb4487-899a-4c3b-b648-557870c7e756 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:00.485772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb4487-899a-4c3b-b648-557870c7e756 None admin] RBAC: Adding query filter params (role.id=89cf5ed93cc642588a54639342c59ea6, scope.project.id=0370c388ce4e42d2ad10ebca467e24dc, user.id=0c802f73a7a64d1b862c3589c9d42f5b) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:00.486085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb4487-899a-4c3b-b648-557870c7e756 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:00.487557 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb4487-899a-4c3b-b648-557870c7e756 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:00.494858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 45/90] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:19:00 2022] GET /identity/v3/role_assignments?user.id=0c802f73a7a64d1b862c3589c9d42f5b&scope.project.id=0370c388ce4e42d2ad10ebca467e24dc&role.id=89cf5ed93cc642588a54639342c59ea6 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:00.495822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb4487-899a-4c3b-b648-557870c7e756 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:00.563185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bab51-befe-4526-9ad0-d0de45be4ed4 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:00.607856 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51b9a2-a848-4b7c-a21f-5e6a189ec72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:00.608791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51b9a2-a848-4b7c-a21f-5e6a189ec72d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:00.609468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51b9a2-a848-4b7c-a21f-5e6a189ec72d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:00.609895 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51b9a2-a848-4b7c-a21f-5e6a189ec72d None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:00.611279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51b9a2-a848-4b7c-a21f-5e6a189ec72d None admin] RBAC: Authorizing `identity:get_project(project_id=0370c388ce4e42d2ad10ebca467e24dc)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:00.614546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51b9a2-a848-4b7c-a21f-5e6a189ec72d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:00.616807 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 46/91] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:00 2022] GET /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:00.618231 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51b9a2-a848-4b7c-a21f-5e6a189ec72d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:00.626136 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb4487-899a-4c3b-b648-557870c7e756 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:00.707729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:00.725329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:00.736714 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cde4f6-f12c-4c1d-9b40-76036f8f94b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:00.737380 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cde4f6-f12c-4c1d-9b40-76036f8f94b6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:00.737664 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cde4f6-f12c-4c1d-9b40-76036f8f94b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:00.737943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cde4f6-f12c-4c1d-9b40-76036f8f94b6 None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:00.740724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cde4f6-f12c-4c1d-9b40-76036f8f94b6 None admin] RBAC: Authorizing `identity:get_project(project_id=0370c388ce4e42d2ad10ebca467e24dc)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:00.742890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cde4f6-f12c-4c1d-9b40-76036f8f94b6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:00.744496 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 46/92] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:00 2022] GET /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:00.746146 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cde4f6-f12c-4c1d-9b40-76036f8f94b6 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:00.758883 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51b9a2-a848-4b7c-a21f-5e6a189ec72d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:00.810366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14443aff-a417-4387-a2e8-d8faaae68e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:00.811453 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14443aff-a417-4387-a2e8-d8faaae68e7e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:00.811887 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14443aff-a417-4387-a2e8-d8faaae68e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:00.812763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14443aff-a417-4387-a2e8-d8faaae68e7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:00.905273 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14443aff-a417-4387-a2e8-d8faaae68e7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:00.908312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14443aff-a417-4387-a2e8-d8faaae68e7e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:00.973725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cde4f6-f12c-4c1d-9b40-76036f8f94b6 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:01.008312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:01.047921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde61ac4-0377-4e41-8b87-3577765e6aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:01.048800 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde61ac4-0377-4e41-8b87-3577765e6aca None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:01.049228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde61ac4-0377-4e41-8b87-3577765e6aca None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:01.049673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde61ac4-0377-4e41-8b87-3577765e6aca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:01.050592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde61ac4-0377-4e41-8b87-3577765e6aca None admin] RBAC: Adding query filter params (role.id=322da7bd64024770a64bdb73e5c54719, scope.project.id=0370c388ce4e42d2ad10ebca467e24dc, user.id=b4cb9e8f0d014047a2ac687139b6261f) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:01.053810 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 47/93] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:01.054456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde61ac4-0377-4e41-8b87-3577765e6aca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:01.055000 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14443aff-a417-4387-a2e8-d8faaae68e7e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:01.055955 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde61ac4-0377-4e41-8b87-3577765e6aca None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:01.061919 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 47/94] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:19:00 2022] GET /identity/v3/role_assignments?user.id=b4cb9e8f0d014047a2ac687139b6261f&scope.project.id=0370c388ce4e42d2ad10ebca467e24dc&role.id=322da7bd64024770a64bdb73e5c54719 => generated 259 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:01.072992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde61ac4-0377-4e41-8b87-3577765e6aca None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:01.198180 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:01.214128 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14443aff-a417-4387-a2e8-d8faaae68e7e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:01.264281 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde61ac4-0377-4e41-8b87-3577765e6aca None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:01.322747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb99803-6676-4764-af14-f65af315efb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:01.323379 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb99803-6676-4764-af14-f65af315efb9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:01.323665 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb99803-6676-4764-af14-f65af315efb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:01.323961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb99803-6676-4764-af14-f65af315efb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:01.324714 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb99803-6676-4764-af14-f65af315efb9 None admin] RBAC: Adding query filter params (role.id=578c9a17c0fa434f9602f5d2b3648785, scope.project.id=0370c388ce4e42d2ad10ebca467e24dc, user.id=0c802f73a7a64d1b862c3589c9d42f5b) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:01.325027 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb99803-6676-4764-af14-f65af315efb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:01.326493 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb99803-6676-4764-af14-f65af315efb9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:01.339623 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d89707-8ffc-4f54-a1e4-3bee5cb7131f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:01.340308 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d89707-8ffc-4f54-a1e4-3bee5cb7131f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:01.340589 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d89707-8ffc-4f54-a1e4-3bee5cb7131f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:01.340863 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d89707-8ffc-4f54-a1e4-3bee5cb7131f None admin] PATH_INFO: `/v3/projects/5355bb857aa0482399be52bcee61d585` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:01.342010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d89707-8ffc-4f54-a1e4-3bee5cb7131f None admin] RBAC: Authorizing `identity:get_project(project_id=5355bb857aa0482399be52bcee61d585)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:01.343470 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d89707-8ffc-4f54-a1e4-3bee5cb7131f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:01.345146 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 48/95] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:01 2022] GET /identity/v3/projects/5355bb857aa0482399be52bcee61d585 => generated 312 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:01.347386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 48/96] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:19:01 2022] GET /identity/v3/role_assignments?user.id=0c802f73a7a64d1b862c3589c9d42f5b&scope.project.id=0370c388ce4e42d2ad10ebca467e24dc&role.id=578c9a17c0fa434f9602f5d2b3648785 => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:01.353060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d89707-8ffc-4f54-a1e4-3bee5cb7131f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:01.506046 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d89707-8ffc-4f54-a1e4-3bee5cb7131f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:01.540541 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:01.551870 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c857844-354f-4019-9911-0b0fbd8dc8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:01.552532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c857844-354f-4019-9911-0b0fbd8dc8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:01.552812 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c857844-354f-4019-9911-0b0fbd8dc8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:01.553090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c857844-354f-4019-9911-0b0fbd8dc8e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:01.553836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c857844-354f-4019-9911-0b0fbd8dc8e9 None admin] RBAC: Adding query filter params (role.id=89cf5ed93cc642588a54639342c59ea6, scope.project.id=5355bb857aa0482399be52bcee61d585, user.id=0c802f73a7a64d1b862c3589c9d42f5b) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:01.554339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c857844-354f-4019-9911-0b0fbd8dc8e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:01.556266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c857844-354f-4019-9911-0b0fbd8dc8e9 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:01.568086 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 49/97] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:19:01 2022] GET /identity/v3/role_assignments?user.id=0c802f73a7a64d1b862c3589c9d42f5b&scope.project.id=5355bb857aa0482399be52bcee61d585&role.id=89cf5ed93cc642588a54639342c59ea6 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:01.808485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddd1de-88f3-4445-bfe4-d93a3ddfb815 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:01.808986 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddd1de-88f3-4445-bfe4-d93a3ddfb815 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:01.809282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddd1de-88f3-4445-bfe4-d93a3ddfb815 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:01.810911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 49/98] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:01.816478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de52c4-0233-45fe-90f2-d2dbd8ed50bc None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:01.816891 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de52c4-0233-45fe-90f2-d2dbd8ed50bc None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:01.817280 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de52c4-0233-45fe-90f2-d2dbd8ed50bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:01.836928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-a0de52c4-0233-45fe-90f2-d2dbd8ed50bc None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:01.838129 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a0de52c4-0233-45fe-90f2-d2dbd8ed50bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:01.915970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 50/99] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:02.177105 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75524bae-d738-4c7e-a5ab-9226a0d50b0e None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:02.177484 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75524bae-d738-4c7e-a5ab-9226a0d50b0e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:02.177600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75524bae-d738-4c7e-a5ab-9226a0d50b0e None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:02.178942 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 50/100] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:02.184407 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c714a64-e3bd-4ede-bccb-5128bb1be8d4 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:02.184856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c714a64-e3bd-4ede-bccb-5128bb1be8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:02.185034 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c714a64-e3bd-4ede-bccb-5128bb1be8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:02.213050 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-7c714a64-e3bd-4ede-bccb-5128bb1be8d4 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:02.214278 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7c714a64-e3bd-4ede-bccb-5128bb1be8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:02.225911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75524bae-d738-4c7e-a5ab-9226a0d50b0e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:02.327984 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 51/101] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:02.344302 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:02.461756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75524bae-d738-4c7e-a5ab-9226a0d50b0e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:02.514782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de853748-fa92-4db3-806c-7819041e3322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:02.515485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de853748-fa92-4db3-806c-7819041e3322 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:02.515778 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de853748-fa92-4db3-806c-7819041e3322 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:02.516095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de853748-fa92-4db3-806c-7819041e3322 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:02.532653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de853748-fa92-4db3-806c-7819041e3322 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:02.539038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-de853748-fa92-4db3-806c-7819041e3322 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:19:02.539038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:02.539038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:02.539038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:02.539038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:02.539038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:02.539038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:02.539038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:02.539038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:02.539038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:02.539038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:02.539038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:02.539038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:02.539038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:02.539038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:02.539038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:02.539863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:02.539863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:02.539863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:02.539863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:02.539863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:02.539863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:02.539863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:02.539863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:02.539863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:02.539863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:02.539863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:02.539863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:02.539863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:02.539863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:02.539863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:02.539863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:02.540719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:02.540719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:02.540719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:02.540719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:02.540719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:02.540719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:02.540719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:02.540719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:02.540719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:02.540719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:02.540719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:19:02.540719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:02.540719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 51/102] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:19:02.549037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c714a64-e3bd-4ede-bccb-5128bb1be8d4 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:02.630322 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:02.691134 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de853748-fa92-4db3-806c-7819041e3322 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:02.698314 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c714a64-e3bd-4ede-bccb-5128bb1be8d4 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:02.740379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e79ec-f9e1-4920-9734-a60cec84462b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:02.740379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e79ec-f9e1-4920-9734-a60cec84462b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:02.740379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e79ec-f9e1-4920-9734-a60cec84462b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:02.740379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e79ec-f9e1-4920-9734-a60cec84462b None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:02.740936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e79ec-f9e1-4920-9734-a60cec84462b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:02.740936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e79ec-f9e1-4920-9734-a60cec84462b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:02.740936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e79ec-f9e1-4920-9734-a60cec84462b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:02.747564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 52/103] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:02.756110 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e79ec-f9e1-4920-9734-a60cec84462b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:02.871578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e79ec-f9e1-4920-9734-a60cec84462b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:02.888371 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:02.924354 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be9769-e8b8-4034-880a-2a8740daa2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:02.925614 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be9769-e8b8-4034-880a-2a8740daa2fc None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:02.925892 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be9769-e8b8-4034-880a-2a8740daa2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:02.926747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be9769-e8b8-4034-880a-2a8740daa2fc None admin] PATH_INFO: `/v3/users/8bd7790fcaab414d8cd6cab1d09660a1` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:02.945151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de853748-fa92-4db3-806c-7819041e3322 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:02.948304 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:02.953203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be9769-e8b8-4034-880a-2a8740daa2fc None admin] RBAC: Authorizing `identity:get_user(user_id=8bd7790fcaab414d8cd6cab1d09660a1)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:02.955202 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be9769-e8b8-4034-880a-2a8740daa2fc None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:02.957470 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 53/104] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:02 2022] GET /identity/v3/users/8bd7790fcaab414d8cd6cab1d09660a1 => generated 289 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:02.963408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be9769-e8b8-4034-880a-2a8740daa2fc None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:03.017811 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cddddcf-48a7-49d7-9d38-4350b100ed8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:03.018471 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddddcf-48a7-49d7-9d38-4350b100ed8e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:03.018766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddddcf-48a7-49d7-9d38-4350b100ed8e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:03.019222 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddddcf-48a7-49d7-9d38-4350b100ed8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:03.021479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cddddcf-48a7-49d7-9d38-4350b100ed8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:03.024265 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cddddcf-48a7-49d7-9d38-4350b100ed8e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:03.026281 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 52/105] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:19:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:03.034181 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cddddcf-48a7-49d7-9d38-4350b100ed8e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:03.145656 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be9769-e8b8-4034-880a-2a8740daa2fc None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:03.204771 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c129b-f32c-4908-8bb0-e271190abcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:03.205686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c129b-f32c-4908-8bb0-e271190abcdd None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:03.205686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c129b-f32c-4908-8bb0-e271190abcdd None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:03.205949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c129b-f32c-4908-8bb0-e271190abcdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:03.247775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:03.278309 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c129b-f32c-4908-8bb0-e271190abcdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:03.280080 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c129b-f32c-4908-8bb0-e271190abcdd None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:03.296482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cddddcf-48a7-49d7-9d38-4350b100ed8e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:03.374033 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f48b7b-2f72-4c87-9649-880dd0dd2719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:03.375058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f48b7b-2f72-4c87-9649-880dd0dd2719 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:03.375666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f48b7b-2f72-4c87-9649-880dd0dd2719 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:03.376186 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f48b7b-2f72-4c87-9649-880dd0dd2719 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:03.377166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f48b7b-2f72-4c87-9649-880dd0dd2719 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:03.378847 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f48b7b-2f72-4c87-9649-880dd0dd2719 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:03.397819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 54/106] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 20:19:03.407375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c129b-f32c-4908-8bb0-e271190abcdd None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:03.431462 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 53/107] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 29 20:19:03 2022] POST /identity/v3/projects => generated 302 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:19:03.503797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4b31e-592e-4f77-812a-70d357f27150 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:03.504161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4b31e-592e-4f77-812a-70d357f27150 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:03.504466 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4b31e-592e-4f77-812a-70d357f27150 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:03.505851 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 54/108] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:19:03.508870 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:03.511680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969cd17-d1ea-46ea-8178-834c4f9f0335 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:03.512049 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969cd17-d1ea-46ea-8178-834c4f9f0335 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:03.512306 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969cd17-d1ea-46ea-8178-834c4f9f0335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:03.549863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4969cd17-d1ea-46ea-8178-834c4f9f0335 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:03.551037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4969cd17-d1ea-46ea-8178-834c4f9f0335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:03.572321 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c129b-f32c-4908-8bb0-e271190abcdd None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:03.592302 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:03.646607 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b14316-fd0f-45bc-a42f-bdc6e76afb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:03.647568 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b14316-fd0f-45bc-a42f-bdc6e76afb2e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:03.647852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b14316-fd0f-45bc-a42f-bdc6e76afb2e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:03.648157 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b14316-fd0f-45bc-a42f-bdc6e76afb2e None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:03.649303 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b14316-fd0f-45bc-a42f-bdc6e76afb2e None admin] RBAC: Authorizing `identity:get_project(project_id=0370c388ce4e42d2ad10ebca467e24dc)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:03.650989 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b14316-fd0f-45bc-a42f-bdc6e76afb2e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:03.654480 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 55/109] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:03 2022] GET /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:03.658786 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b14316-fd0f-45bc-a42f-bdc6e76afb2e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:03.690626 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 55/110] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 29 20:19:03.693025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e2016-eff0-4bc1-9e29-0cf096b85f82 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:03.693316 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e2016-eff0-4bc1-9e29-0cf096b85f82 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:03.693591 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e2016-eff0-4bc1-9e29-0cf096b85f82 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:03.696159 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 56/111] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:03.701317 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47da5d-8797-4603-b2f1-345e720a328f None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:03.701590 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47da5d-8797-4603-b2f1-345e720a328f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:03.701897 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47da5d-8797-4603-b2f1-345e720a328f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:03.722662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ce47da5d-8797-4603-b2f1-345e720a328f None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:03.724033 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ce47da5d-8797-4603-b2f1-345e720a328f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:03.794766 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b14316-fd0f-45bc-a42f-bdc6e76afb2e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:03.799541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 57/112] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:19:03.844165 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144f0b7-f93f-4185-9a1f-c73c73605579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:03.845116 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144f0b7-f93f-4185-9a1f-c73c73605579 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:03.845592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144f0b7-f93f-4185-9a1f-c73c73605579 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:03.846055 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144f0b7-f93f-4185-9a1f-c73c73605579 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:03.848009 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144f0b7-f93f-4185-9a1f-c73c73605579 None admin] RBAC: Adding query filter params (role.id=5ae3132138594bb7a1fbacc9d93198c1, scope.project.id=0370c388ce4e42d2ad10ebca467e24dc, user.id=8bd7790fcaab414d8cd6cab1d09660a1) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:03.849628 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144f0b7-f93f-4185-9a1f-c73c73605579 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:03.851429 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144f0b7-f93f-4185-9a1f-c73c73605579 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:03.857848 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 56/113] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:19:03 2022] GET /identity/v3/role_assignments?user.id=8bd7790fcaab414d8cd6cab1d09660a1&scope.project.id=0370c388ce4e42d2ad10ebca467e24dc&role.id=5ae3132138594bb7a1fbacc9d93198c1 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:04.071782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47da5d-8797-4603-b2f1-345e720a328f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:04.137758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144f0b7-f93f-4185-9a1f-c73c73605579 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:04.142546 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:04.174013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:04.195724 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47da5d-8797-4603-b2f1-345e720a328f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:04.232532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f93645-decf-4ee6-8590-01470984815b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:04.233618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f93645-decf-4ee6-8590-01470984815b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:04.233969 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f93645-decf-4ee6-8590-01470984815b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:04.234310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f93645-decf-4ee6-8590-01470984815b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:04.246802 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f93645-decf-4ee6-8590-01470984815b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:04.252321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-57f93645-decf-4ee6-8590-01470984815b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:19:04.252321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:04.252321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:04.252321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:04.252321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:04.252321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:04.252321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:04.252321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:04.252321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:04.252321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:04.252321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:04.252321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:04.252321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:04.252321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:04.252321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:04.252321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:04.253402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:04.253402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:04.253402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:04.253402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:04.253402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:04.253402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:04.253402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:04.253402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:04.253402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:04.253402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:04.253402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:04.253402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:04.253402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:04.253402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:04.253402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:04.253402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:04.254559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:04.254559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:04.254559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:04.254559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:04.254559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:04.254559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:04.254559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:04.254559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:04.254559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:04.254559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:04.254559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:19:04.254559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:04.254559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 58/114] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:19:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:04.259896 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f93645-decf-4ee6-8590-01470984815b None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:04.370980 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f93645-decf-4ee6-8590-01470984815b None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:04.385226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144f0b7-f93f-4185-9a1f-c73c73605579 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:04.411724 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677ec46-d69b-427f-b418-98a1b9490467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:04.412318 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677ec46-d69b-427f-b418-98a1b9490467 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:04.412604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677ec46-d69b-427f-b418-98a1b9490467 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:04.413049 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677ec46-d69b-427f-b418-98a1b9490467 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:04.413739 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677ec46-d69b-427f-b418-98a1b9490467 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:04.414178 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677ec46-d69b-427f-b418-98a1b9490467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:04.415706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677ec46-d69b-427f-b418-98a1b9490467 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:04.427637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 59/115] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:04.433003 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677ec46-d69b-427f-b418-98a1b9490467 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:04.437208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:04.509254 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ddb8d4-371a-476a-9a7a-76d7cdc0b96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:04.509919 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddb8d4-371a-476a-9a7a-76d7cdc0b96d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:04.510203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddb8d4-371a-476a-9a7a-76d7cdc0b96d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:04.510478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddb8d4-371a-476a-9a7a-76d7cdc0b96d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:04.529110 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ddb8d4-371a-476a-9a7a-76d7cdc0b96d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:04.534758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-49ddb8d4-371a-476a-9a7a-76d7cdc0b96d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:04.534758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:04.534758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:04.534758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:04.534758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:04.534758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:04.534758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:04.534758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:04.534758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:04.534758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:04.534758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:04.534758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:04.534758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:04.534758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:04.534758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:04.534758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:04.535706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:04.535706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:04.535706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:04.535706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:04.535706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:04.535706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:04.535706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:04.535706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:04.535706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:04.535706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:04.535706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:04.535706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:04.535706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:04.535706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:04.535706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:04.535706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:04.537090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:04.537090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:04.537090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:04.537090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:04.537090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:04.537090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:04.537090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:04.537090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:04.537090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:04.537090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:04.537090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:04.537090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:04.541506 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 57/116] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:19:04.546197 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677ec46-d69b-427f-b418-98a1b9490467 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:04.547813 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ddb8d4-371a-476a-9a7a-76d7cdc0b96d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:04.581888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf31203-6a8c-4f07-99d4-85259e28ea7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:04.582533 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf31203-6a8c-4f07-99d4-85259e28ea7a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:04.583384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf31203-6a8c-4f07-99d4-85259e28ea7a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:04.583665 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf31203-6a8c-4f07-99d4-85259e28ea7a None admin] PATH_INFO: `/v3/users/b4cb9e8f0d014047a2ac687139b6261f` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:04.587308 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf31203-6a8c-4f07-99d4-85259e28ea7a None admin] RBAC: Authorizing `identity:get_user(user_id=b4cb9e8f0d014047a2ac687139b6261f)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:04.588791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf31203-6a8c-4f07-99d4-85259e28ea7a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:04.590384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 60/117] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:04 2022] GET /identity/v3/users/b4cb9e8f0d014047a2ac687139b6261f => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:04.599950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf31203-6a8c-4f07-99d4-85259e28ea7a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:04.666573 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:04.740249 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ddb8d4-371a-476a-9a7a-76d7cdc0b96d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:04.743136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:04.801631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf31203-6a8c-4f07-99d4-85259e28ea7a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:04.810768 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b5ec0-d7f6-49d9-9d00-3ef23ba7385f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:04.811384 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b5ec0-d7f6-49d9-9d00-3ef23ba7385f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:04.811660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b5ec0-d7f6-49d9-9d00-3ef23ba7385f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:04.811946 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b5ec0-d7f6-49d9-9d00-3ef23ba7385f None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:04.812628 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b5ec0-d7f6-49d9-9d00-3ef23ba7385f None admin] RBAC: Adding query filter params (name=member) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:04.812927 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b5ec0-d7f6-49d9-9d00-3ef23ba7385f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:04.814303 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b5ec0-d7f6-49d9-9d00-3ef23ba7385f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:04.832783 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 58/118] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:04.838711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b5ec0-d7f6-49d9-9d00-3ef23ba7385f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:04.869472 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692d704-e500-420e-bd45-94c586a6687f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:04.870119 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692d704-e500-420e-bd45-94c586a6687f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:04.870401 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692d704-e500-420e-bd45-94c586a6687f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:04.870677 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692d704-e500-420e-bd45-94c586a6687f None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:04.873194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692d704-e500-420e-bd45-94c586a6687f None admin] RBAC: Authorizing `identity:get_project(project_id=0370c388ce4e42d2ad10ebca467e24dc)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:04.875565 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692d704-e500-420e-bd45-94c586a6687f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:04.877513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 61/119] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:04 2022] GET /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:04.883561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692d704-e500-420e-bd45-94c586a6687f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:05.006135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b5ec0-d7f6-49d9-9d00-3ef23ba7385f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:05.014692 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692d704-e500-420e-bd45-94c586a6687f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:05.033674 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:05.048673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8aaf58-b7a2-4f5f-9884-fb720564c4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:05.049352 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8aaf58-b7a2-4f5f-9884-fb720564c4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:05.049633 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8aaf58-b7a2-4f5f-9884-fb720564c4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:05.049910 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8aaf58-b7a2-4f5f-9884-fb720564c4d3 None admin] PATH_INFO: `/v3/users/0c802f73a7a64d1b862c3589c9d42f5b` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:05.051485 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8aaf58-b7a2-4f5f-9884-fb720564c4d3 None admin] RBAC: Authorizing `identity:get_user(user_id=0c802f73a7a64d1b862c3589c9d42f5b)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:05.053765 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8aaf58-b7a2-4f5f-9884-fb720564c4d3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:05.055641 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 59/120] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:04 2022] GET /identity/v3/users/0c802f73a7a64d1b862c3589c9d42f5b => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:05.056343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a66a77-07ec-4d6a-8827-748e5fb0780c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:05.056845 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d3626-2289-4851-a754-2f64814b37c1 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:05.057126 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d3626-2289-4851-a754-2f64814b37c1 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:05.057397 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d3626-2289-4851-a754-2f64814b37c1 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:05.057888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a66a77-07ec-4d6a-8827-748e5fb0780c None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:05.058364 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a66a77-07ec-4d6a-8827-748e5fb0780c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:05.058606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 60/121] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:05.059079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a66a77-07ec-4d6a-8827-748e5fb0780c None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc/users/b4cb9e8f0d014047a2ac687139b6261f/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:05.064735 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d3626-2289-4851-a754-2f64814b37c1 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:05.065844 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a66a77-07ec-4d6a-8827-748e5fb0780c None admin] RBAC: Authorizing `identity:create_grant(project_id=0370c388ce4e42d2ad10ebca467e24dc, user_id=b4cb9e8f0d014047a2ac687139b6261f, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:05.068748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a66a77-07ec-4d6a-8827-748e5fb0780c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:05.086866 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:05.090615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 62/122] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:19:04 2022] PUT /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc/users/b4cb9e8f0d014047a2ac687139b6261f/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:19:05.091924 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7dc74-3aa9-4fd5-b016-bc1d3036372e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:05.092254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7dc74-3aa9-4fd5-b016-bc1d3036372e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:05.093180 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7dc74-3aa9-4fd5-b016-bc1d3036372e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:05.132532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-dec7dc74-3aa9-4fd5-b016-bc1d3036372e None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:05.133782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-dec7dc74-3aa9-4fd5-b016-bc1d3036372e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:05.238886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 63/123] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:05.268119 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d3626-2289-4851-a754-2f64814b37c1 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:05.319783 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8af4d-6e6b-4612-9d34-761cc51e7c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:05.320609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8af4d-6e6b-4612-9d34-761cc51e7c9b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:05.321093 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8af4d-6e6b-4612-9d34-761cc51e7c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:05.321414 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8af4d-6e6b-4612-9d34-761cc51e7c9b None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:05.322602 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8af4d-6e6b-4612-9d34-761cc51e7c9b None admin] RBAC: Authorizing `identity:get_project(project_id=0370c388ce4e42d2ad10ebca467e24dc)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:05.324678 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8af4d-6e6b-4612-9d34-761cc51e7c9b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:05.327227 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 61/124] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:05 2022] GET /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:05.333663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7dc74-3aa9-4fd5-b016-bc1d3036372e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:05.428234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db950a-5fab-4cf1-aa83-8f567ee27d93 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:05.428729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db950a-5fab-4cf1-aa83-8f567ee27d93 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:05.428800 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db950a-5fab-4cf1-aa83-8f567ee27d93 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:05.430262 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 62/125] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:05.441658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12e9c1-4f71-4706-af51-0792b9e52b1f None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:05.441955 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12e9c1-4f71-4706-af51-0792b9e52b1f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:05.442221 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12e9c1-4f71-4706-af51-0792b9e52b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:05.467344 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7dc74-3aa9-4fd5-b016-bc1d3036372e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:05.481573 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1b12e9c1-4f71-4706-af51-0792b9e52b1f None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:05.482780 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1b12e9c1-4f71-4706-af51-0792b9e52b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:05.545825 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:05.548609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:05.561705 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 63/126] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:19:05.581874 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a77aa8-d4f5-474c-9390-c3e704d92842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:05.582889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a77aa8-d4f5-474c-9390-c3e704d92842 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:05.583211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a77aa8-d4f5-474c-9390-c3e704d92842 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:05.583499 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a77aa8-d4f5-474c-9390-c3e704d92842 None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc/users/0c802f73a7a64d1b862c3589c9d42f5b/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:05.589254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a77aa8-d4f5-474c-9390-c3e704d92842 None admin] RBAC: Authorizing `identity:create_grant(project_id=0370c388ce4e42d2ad10ebca467e24dc, user_id=0c802f73a7a64d1b862c3589c9d42f5b, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:05.590743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a77aa8-d4f5-474c-9390-c3e704d92842 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:05.612912 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 64/127] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:19:05 2022] PUT /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc/users/0c802f73a7a64d1b862c3589c9d42f5b/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:19:05.768293 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12e9c1-4f71-4706-af51-0792b9e52b1f None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:05.888813 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12e9c1-4f71-4706-af51-0792b9e52b1f None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:05.925422 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612ae76-270f-4754-b172-421195bc0c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:05.926068 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612ae76-270f-4754-b172-421195bc0c57 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:05.926348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612ae76-270f-4754-b172-421195bc0c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:05.926622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a612ae76-270f-4754-b172-421195bc0c57 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:05.937133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a612ae76-270f-4754-b172-421195bc0c57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:05.941949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-a612ae76-270f-4754-b172-421195bc0c57 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:05.941949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:05.941949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:05.941949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:05.941949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:05.941949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:05.941949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:05.941949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:05.941949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:05.941949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:05.941949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:05.941949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:05.941949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:05.941949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:05.941949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:05.941949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:05.942814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:05.942814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:05.942814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:05.942814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:05.942814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:05.942814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:05.942814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:05.942814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:05.942814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:05.942814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:05.942814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:05.942814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:05.942814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:05.942814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:05.942814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:05.942814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:05.943978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:05.943978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:05.943978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:05.943978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:05.943978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:05.943978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:05.943978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:05.943978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:05.943978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:05.943978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:05.943978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:05.943978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:05.943978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 64/128] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:19:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:05.952203 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a77aa8-d4f5-474c-9390-c3e704d92842 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:05.988183 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612ae76-270f-4754-b172-421195bc0c57 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:06.072595 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:06.101146 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a612ae76-270f-4754-b172-421195bc0c57 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:06.167130 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c317c57-6be0-4bf7-986d-9e493ace3325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:06.167714 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c317c57-6be0-4bf7-986d-9e493ace3325 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:06.168021 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c317c57-6be0-4bf7-986d-9e493ace3325 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:06.168905 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c317c57-6be0-4bf7-986d-9e493ace3325 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:06.193004 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c317c57-6be0-4bf7-986d-9e493ace3325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:06.197933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-6c317c57-6be0-4bf7-986d-9e493ace3325 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:06.197933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:06.197933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:06.197933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:06.197933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:06.197933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:06.197933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:06.197933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:06.197933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:06.197933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:06.197933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:06.197933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:06.197933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:06.197933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:06.197933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:06.197933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:06.198794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:06.198794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:06.198794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:06.198794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:06.198794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:06.198794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:06.198794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:06.198794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:06.198794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:06.198794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:06.198794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:06.198794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:06.198794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:06.198794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:06.198794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:06.198794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:06.200035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:06.200035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:06.200035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:06.200035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:06.200035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:06.200035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:06.200035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:06.200035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:06.200035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:06.200035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:06.200035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:06.200035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:06.200035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 65/129] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:06.200879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccb68d-2438-4aa2-b8b0-7327fe800d1f None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:06.200879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccb68d-2438-4aa2-b8b0-7327fe800d1f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:06.200879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccb68d-2438-4aa2-b8b0-7327fe800d1f None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:06.202273 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 66/130] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:06.207087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccb68d-2438-4aa2-b8b0-7327fe800d1f None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:06.233351 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a77aa8-d4f5-474c-9390-c3e704d92842 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:06.295088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff1529-a54b-4be1-9208-ed86eb9b0108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:06.298731 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff1529-a54b-4be1-9208-ed86eb9b0108 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:06.300108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff1529-a54b-4be1-9208-ed86eb9b0108 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:06.300704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff1529-a54b-4be1-9208-ed86eb9b0108 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:06.303155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff1529-a54b-4be1-9208-ed86eb9b0108 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:06.303788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff1529-a54b-4be1-9208-ed86eb9b0108 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:06.306583 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff1529-a54b-4be1-9208-ed86eb9b0108 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:06.323698 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 65/131] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:19:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:06.325400 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ade9d2-a52d-4db1-9a02-d12ce67033ae None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:06.326802 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ade9d2-a52d-4db1-9a02-d12ce67033ae None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:06.327370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ade9d2-a52d-4db1-9a02-d12ce67033ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:06.363298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-c2ade9d2-a52d-4db1-9a02-d12ce67033ae None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:06.365793 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c2ade9d2-a52d-4db1-9a02-d12ce67033ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:06.386933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccb68d-2438-4aa2-b8b0-7327fe800d1f None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:06.407766 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:06.446656 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:06.449184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c026488-d20b-4c3e-9fe6-6c4e8e3dfaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:06.449956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c026488-d20b-4c3e-9fe6-6c4e8e3dfaab None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:06.450470 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c026488-d20b-4c3e-9fe6-6c4e8e3dfaab None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:06.450844 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c026488-d20b-4c3e-9fe6-6c4e8e3dfaab None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:06.451994 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c026488-d20b-4c3e-9fe6-6c4e8e3dfaab None admin] RBAC: Adding query filter params (name=member) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:06.452403 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c026488-d20b-4c3e-9fe6-6c4e8e3dfaab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:06.457192 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c026488-d20b-4c3e-9fe6-6c4e8e3dfaab None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:06.467358 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 67/132] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:06.468745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c026488-d20b-4c3e-9fe6-6c4e8e3dfaab None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:06.486919 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 66/133] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:06.487862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ade9d2-a52d-4db1-9a02-d12ce67033ae None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:06.587129 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c026488-d20b-4c3e-9fe6-6c4e8e3dfaab None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:06.638236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5552d-08e6-413c-9e15-55d257f856c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:06.639578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5552d-08e6-413c-9e15-55d257f856c9 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:06.640142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5552d-08e6-413c-9e15-55d257f856c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:06.640614 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5552d-08e6-413c-9e15-55d257f856c9 None admin] PATH_INFO: `/v3/users/0c802f73a7a64d1b862c3589c9d42f5b` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:06.641975 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5552d-08e6-413c-9e15-55d257f856c9 None admin] RBAC: Authorizing `identity:get_user(user_id=0c802f73a7a64d1b862c3589c9d42f5b)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:06.643775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5552d-08e6-413c-9e15-55d257f856c9 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:06.645599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 68/134] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:06 2022] GET /identity/v3/users/0c802f73a7a64d1b862c3589c9d42f5b => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:06.654042 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5552d-08e6-413c-9e15-55d257f856c9 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:06.688686 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ade9d2-a52d-4db1-9a02-d12ce67033ae None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:06.719312 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:06.783923 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204241a-e1c8-4b9e-847e-15c97b5ab446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:06.784866 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204241a-e1c8-4b9e-847e-15c97b5ab446 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:06.785310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204241a-e1c8-4b9e-847e-15c97b5ab446 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:06.785730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204241a-e1c8-4b9e-847e-15c97b5ab446 None admin] PATH_INFO: `/v3/users/0c802f73a7a64d1b862c3589c9d42f5b` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:06.787316 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204241a-e1c8-4b9e-847e-15c97b5ab446 None admin] RBAC: Authorizing `identity:get_user(user_id=0c802f73a7a64d1b862c3589c9d42f5b)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:06.789008 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204241a-e1c8-4b9e-847e-15c97b5ab446 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:06.790892 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 67/135] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:06 2022] GET /identity/v3/users/0c802f73a7a64d1b862c3589c9d42f5b => generated 275 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:06.795668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5552d-08e6-413c-9e15-55d257f856c9 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:06.796968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204241a-e1c8-4b9e-847e-15c97b5ab446 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:06.830025 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f400dc8-f1a4-4ad3-8869-44153315421e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:06.830815 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f400dc8-f1a4-4ad3-8869-44153315421e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:06.831198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f400dc8-f1a4-4ad3-8869-44153315421e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:06.831574 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f400dc8-f1a4-4ad3-8869-44153315421e None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:06.833004 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f400dc8-f1a4-4ad3-8869-44153315421e None admin] RBAC: Authorizing `identity:get_project(project_id=0370c388ce4e42d2ad10ebca467e24dc)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:06.835206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f400dc8-f1a4-4ad3-8869-44153315421e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:06.837028 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 69/136] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:06 2022] GET /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:06.842860 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f400dc8-f1a4-4ad3-8869-44153315421e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:06.917798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:06.955982 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f400dc8-f1a4-4ad3-8869-44153315421e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:06.976601 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:06.991069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb0d26-34eb-4627-82c0-5c598737def3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:06.992917 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb0d26-34eb-4627-82c0-5c598737def3 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:06.993210 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb0d26-34eb-4627-82c0-5c598737def3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:06.993492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb0d26-34eb-4627-82c0-5c598737def3 None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc/users/0c802f73a7a64d1b862c3589c9d42f5b/roles/578c9a17c0fa434f9602f5d2b3648785` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:06.998938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb0d26-34eb-4627-82c0-5c598737def3 None admin] RBAC: Authorizing `identity:create_grant(project_id=0370c388ce4e42d2ad10ebca467e24dc, user_id=0c802f73a7a64d1b862c3589c9d42f5b, role_id=578c9a17c0fa434f9602f5d2b3648785)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:07.001498 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb0d26-34eb-4627-82c0-5c598737def3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:07.004244 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204241a-e1c8-4b9e-847e-15c97b5ab446 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:07.018492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 70/137] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:19:06 2022] PUT /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc/users/0c802f73a7a64d1b862c3589c9d42f5b/roles/578c9a17c0fa434f9602f5d2b3648785 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:19:07.020336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f64527-64a3-4500-b40e-71f0ee0b323c None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:07.020945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f64527-64a3-4500-b40e-71f0ee0b323c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:07.021993 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f64527-64a3-4500-b40e-71f0ee0b323c None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:07.023528 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 71/138] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:07.024678 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f64527-64a3-4500-b40e-71f0ee0b323c None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:07.070502 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787e4f9-01dc-42ff-aec7-d6975b5c32ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:07.071170 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787e4f9-01dc-42ff-aec7-d6975b5c32ef None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:07.071458 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787e4f9-01dc-42ff-aec7-d6975b5c32ef None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:07.071739 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787e4f9-01dc-42ff-aec7-d6975b5c32ef None admin] PATH_INFO: `/v3/projects/5355bb857aa0482399be52bcee61d585` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:07.076819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787e4f9-01dc-42ff-aec7-d6975b5c32ef None admin] RBAC: Authorizing `identity:get_project(project_id=5355bb857aa0482399be52bcee61d585)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:07.078400 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787e4f9-01dc-42ff-aec7-d6975b5c32ef None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:07.080108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 68/139] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:06 2022] GET /identity/v3/projects/5355bb857aa0482399be52bcee61d585 => generated 312 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:07.081433 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2924a-d003-4534-8fa4-25b13231908d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:07.081716 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2924a-d003-4534-8fa4-25b13231908d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:07.082002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2924a-d003-4534-8fa4-25b13231908d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:07.125368 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-37e2924a-d003-4534-8fa4-25b13231908d None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:07.126600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-37e2924a-d003-4534-8fa4-25b13231908d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:07.149775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f64527-64a3-4500-b40e-71f0ee0b323c None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:07.188309 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3860e-3692-4354-8eae-ee1023e8b14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:07.189366 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3860e-3692-4354-8eae-ee1023e8b14f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:07.189836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3860e-3692-4354-8eae-ee1023e8b14f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:07.190335 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3860e-3692-4354-8eae-ee1023e8b14f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:07.191664 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b3860e-3692-4354-8eae-ee1023e8b14f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:07.193639 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b3860e-3692-4354-8eae-ee1023e8b14f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:07.195864 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 72/140] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:19:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:07.201519 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3860e-3692-4354-8eae-ee1023e8b14f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:07.269564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 69/141] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:07.270429 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2924a-d003-4534-8fa4-25b13231908d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:07.292385 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:07.317390 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b3860e-3692-4354-8eae-ee1023e8b14f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:07.335399 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:07.358765 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d120b-3b1c-4666-8b5a-84a3c473efc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:07.359599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d120b-3b1c-4666-8b5a-84a3c473efc7 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:07.360009 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d120b-3b1c-4666-8b5a-84a3c473efc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:07.360916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d120b-3b1c-4666-8b5a-84a3c473efc7 None admin] PATH_INFO: `/v3/projects/5355bb857aa0482399be52bcee61d585/users/0c802f73a7a64d1b862c3589c9d42f5b/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:07.363768 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d120b-3b1c-4666-8b5a-84a3c473efc7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5355bb857aa0482399be52bcee61d585, user_id=0c802f73a7a64d1b862c3589c9d42f5b, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:07.365435 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d120b-3b1c-4666-8b5a-84a3c473efc7 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:07.383970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 73/142] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:19:07 2022] PUT /identity/v3/projects/5355bb857aa0482399be52bcee61d585/users/0c802f73a7a64d1b862c3589c9d42f5b/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:19:07.404830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2924a-d003-4534-8fa4-25b13231908d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:07.445721 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fc678-dd8a-4235-9065-10323a6c9d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:07.446429 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fc678-dd8a-4235-9065-10323a6c9d61 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:07.446717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fc678-dd8a-4235-9065-10323a6c9d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:07.446996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fc678-dd8a-4235-9065-10323a6c9d61 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:07.447783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5fc678-dd8a-4235-9065-10323a6c9d61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:07.449482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5fc678-dd8a-4235-9065-10323a6c9d61 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:07.475964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 70/143] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 29 20:19:07 2022] POST /identity/v3/users => generated 283 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:19:07.780019 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d120b-3b1c-4666-8b5a-84a3c473efc7 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:07.783115 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fe280-45c3-40d1-a023-e18caa1d320b None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:07.783523 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fe280-45c3-40d1-a023-e18caa1d320b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:07.783903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fe280-45c3-40d1-a023-e18caa1d320b None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:07.785422 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 71/144] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:19:07.791138 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66d914-8eb7-4951-ba0c-e3c43bd1c27b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:07.791548 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66d914-8eb7-4951-ba0c-e3c43bd1c27b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:07.791956 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66d914-8eb7-4951-ba0c-e3c43bd1c27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:07.817036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-1c66d914-8eb7-4951-ba0c-e3c43bd1c27b None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:07.818920 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1c66d914-8eb7-4951-ba0c-e3c43bd1c27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:07.888338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:07.891665 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d120b-3b1c-4666-8b5a-84a3c473efc7 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:07.900074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 72/145] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:19:07.943532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb24be9-837f-40c3-bcc6-cfadff8c005f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:07.944205 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb24be9-837f-40c3-bcc6-cfadff8c005f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:07.944483 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb24be9-837f-40c3-bcc6-cfadff8c005f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:07.944751 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb24be9-837f-40c3-bcc6-cfadff8c005f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:07.961789 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb24be9-837f-40c3-bcc6-cfadff8c005f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:07.973077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-bdb24be9-837f-40c3-bcc6-cfadff8c005f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:19:07.973077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:07.973077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:07.973077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:07.973077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:07.973077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:07.973077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:07.973077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:07.973077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:07.973077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:07.973077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:07.973077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:07.973077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:07.973077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:07.973077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:07.973077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:07.973916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:07.973916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:07.973916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:07.973916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:07.973916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:07.973916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:07.973916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:07.973916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:07.973916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:07.973916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:07.973916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:07.973916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:07.973916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:07.973916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:07.973916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:07.973916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:07.974769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:07.974769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:07.974769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:07.974769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:07.974769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:07.974769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:07.974769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:07.974769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:07.974769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:07.974769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:07.974769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:19:07.974769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:07.974769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 74/146] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:07.980310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66d914-8eb7-4951-ba0c-e3c43bd1c27b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:08.035193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:08.111624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66d914-8eb7-4951-ba0c-e3c43bd1c27b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:08.150254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0f6d2-10ec-4107-a9d4-d5fccace12a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:08.150924 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0f6d2-10ec-4107-a9d4-d5fccace12a9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:08.151211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0f6d2-10ec-4107-a9d4-d5fccace12a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:08.151494 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0f6d2-10ec-4107-a9d4-d5fccace12a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:08.152208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0f6d2-10ec-4107-a9d4-d5fccace12a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:08.152517 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0f6d2-10ec-4107-a9d4-d5fccace12a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:08.154616 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0f6d2-10ec-4107-a9d4-d5fccace12a9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:08.164515 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 73/147] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:08.170555 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb24be9-837f-40c3-bcc6-cfadff8c005f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:08.260102 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0f6d2-10ec-4107-a9d4-d5fccace12a9 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:08.289263 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb24be9-837f-40c3-bcc6-cfadff8c005f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:08.345829 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2cec4-34e0-497f-b179-5a48e34ce768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:08.346477 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2cec4-34e0-497f-b179-5a48e34ce768 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:08.346750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2cec4-34e0-497f-b179-5a48e34ce768 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:08.347016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2cec4-34e0-497f-b179-5a48e34ce768 None admin] PATH_INFO: `/v3/users/8bd7790fcaab414d8cd6cab1d09660a1` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:08.350478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2cec4-34e0-497f-b179-5a48e34ce768 None admin] RBAC: Authorizing `identity:get_user(user_id=8bd7790fcaab414d8cd6cab1d09660a1)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:08.351962 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2cec4-34e0-497f-b179-5a48e34ce768 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:08.353735 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 75/148] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:08 2022] GET /identity/v3/users/8bd7790fcaab414d8cd6cab1d09660a1 => generated 289 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:08.360602 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2cec4-34e0-497f-b179-5a48e34ce768 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:08.407582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:08.454297 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:08.467255 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0f6d2-10ec-4107-a9d4-d5fccace12a9 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:08.537536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2cec4-34e0-497f-b179-5a48e34ce768 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:08.540830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28685692-38b2-4634-bea8-79093c9b92db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:08.541434 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28685692-38b2-4634-bea8-79093c9b92db None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:08.541709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28685692-38b2-4634-bea8-79093c9b92db None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:08.541977 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28685692-38b2-4634-bea8-79093c9b92db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:08.562275 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28685692-38b2-4634-bea8-79093c9b92db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:08.569194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-28685692-38b2-4634-bea8-79093c9b92db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:19:08.569194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:08.569194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:08.569194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:08.569194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:08.569194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:08.569194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:08.569194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:08.569194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:08.569194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:08.569194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:08.569194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:08.569194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:08.569194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:08.569194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:08.569194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:08.570108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:08.570108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:08.570108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:08.570108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:08.570108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:08.570108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:08.570108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:08.570108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:08.570108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:08.570108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:08.570108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:08.570108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:08.570108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:08.570108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:08.570108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:08.570108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:08.570888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:08.570888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:08.570888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:08.570888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:08.570888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:08.570888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:08.570888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:08.570888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:08.570888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:08.570888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:08.570888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:19:08.570888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:08.570888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 74/149] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:19:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:08.576759 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28685692-38b2-4634-bea8-79093c9b92db None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:08.593531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6122a-88fd-443e-97e7-4a8a6635d0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:08.594566 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6122a-88fd-443e-97e7-4a8a6635d0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:08.595207 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6122a-88fd-443e-97e7-4a8a6635d0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:08.595575 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6122a-88fd-443e-97e7-4a8a6635d0b8 None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:08.597049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6122a-88fd-443e-97e7-4a8a6635d0b8 None admin] RBAC: Authorizing `identity:get_project(project_id=0370c388ce4e42d2ad10ebca467e24dc)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:08.599240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6122a-88fd-443e-97e7-4a8a6635d0b8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:08.603402 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 76/150] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:08 2022] GET /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:08.616143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6122a-88fd-443e-97e7-4a8a6635d0b8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:08.727071 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28685692-38b2-4634-bea8-79093c9b92db None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:08.772607 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:08.788916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6122a-88fd-443e-97e7-4a8a6635d0b8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:08.799224 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091dd385-8189-46d6-9318-397cfd22075c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:08.799778 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dd385-8189-46d6-9318-397cfd22075c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:08.800087 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dd385-8189-46d6-9318-397cfd22075c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:08.800362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dd385-8189-46d6-9318-397cfd22075c None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:08.801038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dd385-8189-46d6-9318-397cfd22075c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:08.801339 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dd385-8189-46d6-9318-397cfd22075c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:08.803232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dd385-8189-46d6-9318-397cfd22075c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:08.819283 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 75/151] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:08.820910 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ab061-7022-44ac-8dc1-470f8a7f494d None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:08.821194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ab061-7022-44ac-8dc1-470f8a7f494d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:08.821471 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ab061-7022-44ac-8dc1-470f8a7f494d None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:08.823721 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 76/152] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:08.828276 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ab061-7022-44ac-8dc1-470f8a7f494d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:08.860282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:08.869597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3def9a9-6c1f-454e-ab9f-8c6b27f16856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:08.870531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3def9a9-6c1f-454e-ab9f-8c6b27f16856 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:08.870531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3def9a9-6c1f-454e-ab9f-8c6b27f16856 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:08.870794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3def9a9-6c1f-454e-ab9f-8c6b27f16856 None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc/users/8bd7790fcaab414d8cd6cab1d09660a1/roles/5ae3132138594bb7a1fbacc9d93198c1` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:08.874106 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3def9a9-6c1f-454e-ab9f-8c6b27f16856 None admin] RBAC: Authorizing `identity:create_grant(project_id=0370c388ce4e42d2ad10ebca467e24dc, user_id=8bd7790fcaab414d8cd6cab1d09660a1, role_id=5ae3132138594bb7a1fbacc9d93198c1)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:08.876906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3def9a9-6c1f-454e-ab9f-8c6b27f16856 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:08.900933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 77/153] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:19:08 2022] PUT /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc/users/8bd7790fcaab414d8cd6cab1d09660a1/roles/5ae3132138594bb7a1fbacc9d93198c1 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:19:08.901999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fb6ed-f30d-4af7-8e2e-b0917330d114 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:08.902281 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fb6ed-f30d-4af7-8e2e-b0917330d114 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:08.908606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fb6ed-f30d-4af7-8e2e-b0917330d114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:08.953788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-cb7fb6ed-f30d-4af7-8e2e-b0917330d114 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:08.957131 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-cb7fb6ed-f30d-4af7-8e2e-b0917330d114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:09.075257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ab061-7022-44ac-8dc1-470f8a7f494d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:09.118932 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 78/154] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:09.161715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b929d-4959-4d02-b26f-9897bccff0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:09.162421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b929d-4959-4d02-b26f-9897bccff0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:09.162710 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b929d-4959-4d02-b26f-9897bccff0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:09.163011 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b929d-4959-4d02-b26f-9897bccff0e5 None admin] PATH_INFO: `/v3/users/b4cb9e8f0d014047a2ac687139b6261f` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:09.168880 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b929d-4959-4d02-b26f-9897bccff0e5 None admin] RBAC: Authorizing `identity:get_user(user_id=b4cb9e8f0d014047a2ac687139b6261f)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:09.170523 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b929d-4959-4d02-b26f-9897bccff0e5 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:09.172621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 77/155] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:08 2022] GET /identity/v3/users/b4cb9e8f0d014047a2ac687139b6261f => generated 247 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:09.178178 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fb6ed-f30d-4af7-8e2e-b0917330d114 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:09.283462 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:09.315063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fb6ed-f30d-4af7-8e2e-b0917330d114 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:09.356171 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d600e2-7f03-4772-9f41-f5a652618fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:09.356722 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d600e2-7f03-4772-9f41-f5a652618fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:09.357004 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d600e2-7f03-4772-9f41-f5a652618fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:09.357280 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d600e2-7f03-4772-9f41-f5a652618fe7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:09.439533 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d600e2-7f03-4772-9f41-f5a652618fe7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:09.441059 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d600e2-7f03-4772-9f41-f5a652618fe7 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:09.566688 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:09.588745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 79/156] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:09.597020 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b929d-4959-4d02-b26f-9897bccff0e5 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:09.723596 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:09.727236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d600e2-7f03-4772-9f41-f5a652618fe7 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:09.761091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b929d-4959-4d02-b26f-9897bccff0e5 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:09.857592 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a125f26-a925-42ec-96e4-bfd38c8369ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:09.858258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a125f26-a925-42ec-96e4-bfd38c8369ca None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:09.858646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a125f26-a925-42ec-96e4-bfd38c8369ca None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:09.858930 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a125f26-a925-42ec-96e4-bfd38c8369ca None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:09.860104 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a125f26-a925-42ec-96e4-bfd38c8369ca None admin] RBAC: Authorizing `identity:get_project(project_id=0370c388ce4e42d2ad10ebca467e24dc)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:09.861658 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a125f26-a925-42ec-96e4-bfd38c8369ca None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:09.863422 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 78/157] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:09 2022] GET /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:09.878388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a125f26-a925-42ec-96e4-bfd38c8369ca None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:09.902155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d600e2-7f03-4772-9f41-f5a652618fe7 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:09.982962 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116a5fd-5fbf-4201-bd0f-407080ddfe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:09.983965 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116a5fd-5fbf-4201-bd0f-407080ddfe03 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:09.984454 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116a5fd-5fbf-4201-bd0f-407080ddfe03 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:09.984926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116a5fd-5fbf-4201-bd0f-407080ddfe03 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:10.011110 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a125f26-a925-42ec-96e4-bfd38c8369ca None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:10.015225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116a5fd-5fbf-4201-bd0f-407080ddfe03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:10.020460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-9116a5fd-5fbf-4201-bd0f-407080ddfe03 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:10.020460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:10.020460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:10.020460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:10.020460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:10.020460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:10.020460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:10.020460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:10.020460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:10.020460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:10.020460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:10.020460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:10.020460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:10.020460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:10.020460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:10.020460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:10.021400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:10.021400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:10.021400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:10.021400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:10.021400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:10.021400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:10.021400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:10.021400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:10.021400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:10.021400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:10.021400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:10.021400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:10.021400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:10.021400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:10.021400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:10.021400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:10.022260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:10.022260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:10.022260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:10.022260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:10.022260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:10.022260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:10.022260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:10.022260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:10.022260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:10.022260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:10.022260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:10.022260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:10.024876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 80/158] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:10.034189 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116a5fd-5fbf-4201-bd0f-407080ddfe03 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:10.057075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabbdbc-954f-4af2-8219-8f369f0cb78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:10.057707 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabbdbc-954f-4af2-8219-8f369f0cb78e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:10.057997 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabbdbc-954f-4af2-8219-8f369f0cb78e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:10.058279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabbdbc-954f-4af2-8219-8f369f0cb78e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:10.059185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabbdbc-954f-4af2-8219-8f369f0cb78e None admin] RBAC: Adding query filter params (role.id=322da7bd64024770a64bdb73e5c54719, scope.project.id=0370c388ce4e42d2ad10ebca467e24dc, user.id=b4cb9e8f0d014047a2ac687139b6261f) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:10.059517 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabbdbc-954f-4af2-8219-8f369f0cb78e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:10.063171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabbdbc-954f-4af2-8219-8f369f0cb78e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:10.069323 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 79/159] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:19:09 2022] GET /identity/v3/role_assignments?user.id=b4cb9e8f0d014047a2ac687139b6261f&scope.project.id=0370c388ce4e42d2ad10ebca467e24dc&role.id=322da7bd64024770a64bdb73e5c54719 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:10.128832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:10.185596 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3e846-97aa-4f7f-a415-fc18cdc61826 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:10.185883 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3e846-97aa-4f7f-a415-fc18cdc61826 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:10.186749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3e846-97aa-4f7f-a415-fc18cdc61826 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:10.188392 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 80/160] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:10.195415 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd006b-caed-4935-908c-e4a7202c50ef None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:10.196025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd006b-caed-4935-908c-e4a7202c50ef None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:10.196753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd006b-caed-4935-908c-e4a7202c50ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:10.207503 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116a5fd-5fbf-4201-bd0f-407080ddfe03 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:10.237606 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-5acd006b-caed-4935-908c-e4a7202c50ef None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:10.239197 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5acd006b-caed-4935-908c-e4a7202c50ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:10.265999 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:10.356693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b0730-6153-43d2-a9e6-f8387c16a68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:10.357334 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b0730-6153-43d2-a9e6-f8387c16a68f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:10.357613 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b0730-6153-43d2-a9e6-f8387c16a68f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:10.357885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b0730-6153-43d2-a9e6-f8387c16a68f None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:10.358576 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b0730-6153-43d2-a9e6-f8387c16a68f None admin] RBAC: Adding query filter params (name=member) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:10.359056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b0730-6153-43d2-a9e6-f8387c16a68f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:10.360633 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b0730-6153-43d2-a9e6-f8387c16a68f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:10.364990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 81/161] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:19:10.383157 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 81/162] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:10.388708 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd006b-caed-4935-908c-e4a7202c50ef None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:10.518178 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd006b-caed-4935-908c-e4a7202c50ef None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:10.555258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808e944-796e-4fcb-90be-615a8661edb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:10.556221 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808e944-796e-4fcb-90be-615a8661edb5 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:10.556668 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808e944-796e-4fcb-90be-615a8661edb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:10.557145 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808e944-796e-4fcb-90be-615a8661edb5 None admin] PATH_INFO: `/v3/users/0c802f73a7a64d1b862c3589c9d42f5b` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:10.558441 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6808e944-796e-4fcb-90be-615a8661edb5 None admin] RBAC: Authorizing `identity:get_user(user_id=0c802f73a7a64d1b862c3589c9d42f5b)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:10.560076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6808e944-796e-4fcb-90be-615a8661edb5 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:10.562176 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 82/163] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:10 2022] GET /identity/v3/users/0c802f73a7a64d1b862c3589c9d42f5b => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:10.567604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b0730-6153-43d2-a9e6-f8387c16a68f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:10.710407 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b0730-6153-43d2-a9e6-f8387c16a68f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:10.713458 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:10.726353 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9997531-dd63-4e61-98dc-06a43ca62ae8 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:10.726875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9997531-dd63-4e61-98dc-06a43ca62ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:10.726961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9997531-dd63-4e61-98dc-06a43ca62ae8 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:10.728432 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 83/164] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:19:10.733918 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c62e1-6d73-4f54-8900-1be99575644c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:10.734220 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c62e1-6d73-4f54-8900-1be99575644c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:10.734501 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c62e1-6d73-4f54-8900-1be99575644c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:10.772427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-682c62e1-6d73-4f54-8900-1be99575644c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:10.773869 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-682c62e1-6d73-4f54-8900-1be99575644c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:10.774069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba778f31-0a8a-45bc-a460-7d945a4e56b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:10.774687 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba778f31-0a8a-45bc-a460-7d945a4e56b9 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:10.774975 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba778f31-0a8a-45bc-a460-7d945a4e56b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:10.775252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba778f31-0a8a-45bc-a460-7d945a4e56b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:10.794932 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:10.879346 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 84/165] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:10.914959 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba778f31-0a8a-45bc-a460-7d945a4e56b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:10.917104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba778f31-0a8a-45bc-a460-7d945a4e56b9 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:11.005117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c62e1-6d73-4f54-8900-1be99575644c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:11.041840 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 82/166] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:11.047007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba778f31-0a8a-45bc-a460-7d945a4e56b9 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:11.049911 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:11.214421 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba778f31-0a8a-45bc-a460-7d945a4e56b9 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:11.247804 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c62e1-6d73-4f54-8900-1be99575644c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:11.275268 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0bcbc2-0f0b-4e09-913b-014c87de281e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:11.275960 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bcbc2-0f0b-4e09-913b-014c87de281e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:11.276264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bcbc2-0f0b-4e09-913b-014c87de281e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:11.276538 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bcbc2-0f0b-4e09-913b-014c87de281e None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:11.277878 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0bcbc2-0f0b-4e09-913b-014c87de281e None admin] RBAC: Authorizing `identity:get_project(project_id=0370c388ce4e42d2ad10ebca467e24dc)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:11.279392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0bcbc2-0f0b-4e09-913b-014c87de281e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:11.286887 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 83/167] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:11 2022] GET /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:11.298140 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0bcbc2-0f0b-4e09-913b-014c87de281e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:11.338477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15832500-34c9-4456-81db-ad4c78138690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:11.339394 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15832500-34c9-4456-81db-ad4c78138690 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:11.339833 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15832500-34c9-4456-81db-ad4c78138690 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:11.340358 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15832500-34c9-4456-81db-ad4c78138690 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:11.363380 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15832500-34c9-4456-81db-ad4c78138690 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:11.368903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-15832500-34c9-4456-81db-ad4c78138690 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:11.368903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:11.368903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:11.368903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:11.368903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:11.368903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:11.368903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:11.368903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:11.368903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:11.368903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:11.368903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:11.368903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:11.368903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:11.368903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:11.368903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:11.368903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:11.370126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:11.370126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:11.370126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:11.370126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:11.370126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:11.370126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:11.370126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:11.370126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:11.370126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:11.370126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:11.370126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:11.370126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:11.370126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:11.370126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:11.370126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:11.370126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:11.371086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:11.371086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:11.371086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:11.371086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:11.371086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:11.371086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:11.371086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:11.371086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:11.371086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:11.371086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:11.371086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:11.371086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:11.373275 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 85/168] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:19:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:11.374318 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15832500-34c9-4456-81db-ad4c78138690 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:11.379076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:11.461514 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:11.512766 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0bcbc2-0f0b-4e09-913b-014c87de281e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:11.574182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f1b84-4baf-4fcd-8e3a-143716c2619b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:11.575035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f1b84-4baf-4fcd-8e3a-143716c2619b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:11.575463 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f1b84-4baf-4fcd-8e3a-143716c2619b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:11.575882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f1b84-4baf-4fcd-8e3a-143716c2619b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:11.581324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f1b84-4baf-4fcd-8e3a-143716c2619b None admin] RBAC: Adding query filter params (role.id=89cf5ed93cc642588a54639342c59ea6, scope.project.id=0370c388ce4e42d2ad10ebca467e24dc, user.id=0c802f73a7a64d1b862c3589c9d42f5b) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:11.581832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f1b84-4baf-4fcd-8e3a-143716c2619b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:11.583480 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f1b84-4baf-4fcd-8e3a-143716c2619b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:11.595767 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15832500-34c9-4456-81db-ad4c78138690 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:11.596175 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 84/169] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:19:11 2022] GET /identity/v3/role_assignments?user.id=0c802f73a7a64d1b862c3589c9d42f5b&scope.project.id=0370c388ce4e42d2ad10ebca467e24dc&role.id=89cf5ed93cc642588a54639342c59ea6 => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:11.597302 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f1b84-4baf-4fcd-8e3a-143716c2619b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:11.684138 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc0d16-ea8c-4517-b87a-6e61c76cc369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:11.684798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc0d16-ea8c-4517-b87a-6e61c76cc369 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:11.685080 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc0d16-ea8c-4517-b87a-6e61c76cc369 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:11.685354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc0d16-ea8c-4517-b87a-6e61c76cc369 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:11.716142 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc0d16-ea8c-4517-b87a-6e61c76cc369 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:11.721299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-29dc0d16-ea8c-4517-b87a-6e61c76cc369 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:11.721299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:11.721299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:11.721299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:11.721299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:11.721299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:11.721299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:11.721299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:11.721299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:11.721299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:11.721299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:11.721299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:11.721299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:11.721299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:11.721299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:11.721299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:11.722155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:11.722155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:11.722155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:11.722155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:11.722155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:11.722155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:11.722155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:11.722155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:11.722155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:11.722155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:11.722155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:11.722155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:11.722155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:11.722155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:11.722155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:11.722155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:11.722955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:11.722955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:11.722955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:11.722955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:11.722955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:11.722955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:11.722955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:11.722955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:11.722955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:11.722955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:11.722955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:11.722955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:11.722955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 86/170] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:11.731223 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc0d16-ea8c-4517-b87a-6e61c76cc369 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:11.852714 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f1b84-4baf-4fcd-8e3a-143716c2619b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:11.860229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:11.952756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc0d16-ea8c-4517-b87a-6e61c76cc369 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:11.972657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2e761-da80-44ef-a8f8-d16516238ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:11.973330 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2e761-da80-44ef-a8f8-d16516238ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:11.973564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2e761-da80-44ef-a8f8-d16516238ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:11.973840 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2e761-da80-44ef-a8f8-d16516238ec5 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:11.974537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2e761-da80-44ef-a8f8-d16516238ec5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:11.974836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2e761-da80-44ef-a8f8-d16516238ec5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:11.976957 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e2e761-da80-44ef-a8f8-d16516238ec5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:11.979596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:11.990809 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 85/171] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:19:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:11.996803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2e761-da80-44ef-a8f8-d16516238ec5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:12.020800 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b0c9a-b876-48ab-891f-2536da8b06e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:12.021426 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b0c9a-b876-48ab-891f-2536da8b06e3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:12.021709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b0c9a-b876-48ab-891f-2536da8b06e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:12.022505 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b0c9a-b876-48ab-891f-2536da8b06e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:12.023202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b0c9a-b876-48ab-891f-2536da8b06e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:12.023507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b0c9a-b876-48ab-891f-2536da8b06e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:12.025034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b0c9a-b876-48ab-891f-2536da8b06e3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:12.043961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 87/172] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:12.055713 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b0c9a-b876-48ab-891f-2536da8b06e3 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:12.179367 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2e761-da80-44ef-a8f8-d16516238ec5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:12.219185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b0c9a-b876-48ab-891f-2536da8b06e3 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:12.239520 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57b1cd-3219-40ab-8054-2b6060fc092e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:12.240464 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57b1cd-3219-40ab-8054-2b6060fc092e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:12.240968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57b1cd-3219-40ab-8054-2b6060fc092e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:12.244063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57b1cd-3219-40ab-8054-2b6060fc092e None admin] PATH_INFO: `/v3/users/0c802f73a7a64d1b862c3589c9d42f5b` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:12.245511 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57b1cd-3219-40ab-8054-2b6060fc092e None admin] RBAC: Authorizing `identity:get_user(user_id=0c802f73a7a64d1b862c3589c9d42f5b)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:12.250365 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57b1cd-3219-40ab-8054-2b6060fc092e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:12.252051 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:12.253931 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 86/173] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:11 2022] GET /identity/v3/users/0c802f73a7a64d1b862c3589c9d42f5b => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:12.256205 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067c0b3-5f40-418d-9e53-8201842e6f67 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:12.257455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067c0b3-5f40-418d-9e53-8201842e6f67 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:12.258218 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067c0b3-5f40-418d-9e53-8201842e6f67 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:12.261199 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 87/174] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:12.262666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067c0b3-5f40-418d-9e53-8201842e6f67 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:12.272152 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ddad5-881b-4540-8d88-7337dd3dcd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:12.273327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ddad5-881b-4540-8d88-7337dd3dcd68 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:12.273616 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ddad5-881b-4540-8d88-7337dd3dcd68 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:12.273898 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ddad5-881b-4540-8d88-7337dd3dcd68 None admin] PATH_INFO: `/v3/users/0c802f73a7a64d1b862c3589c9d42f5b` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:12.275815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ddad5-881b-4540-8d88-7337dd3dcd68 None admin] RBAC: Authorizing `identity:get_user(user_id=0c802f73a7a64d1b862c3589c9d42f5b)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:12.277331 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ddad5-881b-4540-8d88-7337dd3dcd68 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:12.284012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 88/175] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:12 2022] GET /identity/v3/users/0c802f73a7a64d1b862c3589c9d42f5b => generated 275 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:12.284012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17bbd1-7b9e-4c5b-be5a-89875f698f6c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:12.284012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17bbd1-7b9e-4c5b-be5a-89875f698f6c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:12.284012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17bbd1-7b9e-4c5b-be5a-89875f698f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:12.311899 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-0c17bbd1-7b9e-4c5b-be5a-89875f698f6c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:12.313220 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0c17bbd1-7b9e-4c5b-be5a-89875f698f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:12.352700 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:12.397443 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067c0b3-5f40-418d-9e53-8201842e6f67 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:12.411788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 89/176] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:12.411788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17bbd1-7b9e-4c5b-be5a-89875f698f6c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:12.468218 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0771a-1718-4768-8dae-b98ddf6d8103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:12.469170 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0771a-1718-4768-8dae-b98ddf6d8103 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:12.469651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0771a-1718-4768-8dae-b98ddf6d8103 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:12.470125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0771a-1718-4768-8dae-b98ddf6d8103 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:12.524319 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17bbd1-7b9e-4c5b-be5a-89875f698f6c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:12.564808 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4414c7f-2b5d-42f8-b135-592b7311e43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:12.565446 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4414c7f-2b5d-42f8-b135-592b7311e43d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:12.565789 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4414c7f-2b5d-42f8-b135-592b7311e43d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:12.566076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4414c7f-2b5d-42f8-b135-592b7311e43d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:12.572290 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:12.625085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf0771a-1718-4768-8dae-b98ddf6d8103 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:12.626772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf0771a-1718-4768-8dae-b98ddf6d8103 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:12.650889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4414c7f-2b5d-42f8-b135-592b7311e43d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:12.652417 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4414c7f-2b5d-42f8-b135-592b7311e43d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:12.746587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:12.770601 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 90/177] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:12.770601 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1359544-49e0-4d96-94cf-c5722c5c680f None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:12.770601 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1359544-49e0-4d96-94cf-c5722c5c680f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:12.770601 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1359544-49e0-4d96-94cf-c5722c5c680f None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:12.770601 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 91/178] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:12.771827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229af3e-16f6-4837-8ee2-bf2615c73602 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:12.771924 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229af3e-16f6-4837-8ee2-bf2615c73602 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:12.772738 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229af3e-16f6-4837-8ee2-bf2615c73602 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:12.774071 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 92/179] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:12.774824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229af3e-16f6-4837-8ee2-bf2615c73602 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:12.798033 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 88/180] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:12.799759 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f05a33-a5f6-4fbb-9f23-9c17e96dc6c0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:12.800265 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f05a33-a5f6-4fbb-9f23-9c17e96dc6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:12.800765 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f05a33-a5f6-4fbb-9f23-9c17e96dc6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:12.804488 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:12.823979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d4f05a33-a5f6-4fbb-9f23-9c17e96dc6c0 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:12.825621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d4f05a33-a5f6-4fbb-9f23-9c17e96dc6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:12.898915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229af3e-16f6-4837-8ee2-bf2615c73602 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:12.915997 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 89/181] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:12.917616 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f6bee-7319-4654-a416-1f2b71633122 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:12.918068 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f6bee-7319-4654-a416-1f2b71633122 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:12.918487 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f6bee-7319-4654-a416-1f2b71633122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:12.946947 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-174f6bee-7319-4654-a416-1f2b71633122 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:12.947567 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfb81b-5fd2-499c-8d63-6f5ced9aa8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:12.948142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-174f6bee-7319-4654-a416-1f2b71633122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:12.948624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfb81b-5fd2-499c-8d63-6f5ced9aa8ec None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:12.949047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfb81b-5fd2-499c-8d63-6f5ced9aa8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:12.949491 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dfb81b-5fd2-499c-8d63-6f5ced9aa8ec None admin] PATH_INFO: `/v3/projects/5355bb857aa0482399be52bcee61d585` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:12.952060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dfb81b-5fd2-499c-8d63-6f5ced9aa8ec None admin] RBAC: Authorizing `identity:get_project(project_id=5355bb857aa0482399be52bcee61d585)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:12.953958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dfb81b-5fd2-499c-8d63-6f5ced9aa8ec None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:12.956407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 93/182] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:12 2022] GET /identity/v3/projects/5355bb857aa0482399be52bcee61d585 => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:12.957408 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfb81b-5fd2-499c-8d63-6f5ced9aa8ec None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:13.028002 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 90/183] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:13.029604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f6bee-7319-4654-a416-1f2b71633122 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:13.060871 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:13.086358 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:13.101352 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dfb81b-5fd2-499c-8d63-6f5ced9aa8ec None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:13.147862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456be928-9f64-463d-8268-e98edb60f310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:13.148881 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456be928-9f64-463d-8268-e98edb60f310 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:13.149212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f6bee-7319-4654-a416-1f2b71633122 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:13.149659 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456be928-9f64-463d-8268-e98edb60f310 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:13.150166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456be928-9f64-463d-8268-e98edb60f310 None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:13.152530 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456be928-9f64-463d-8268-e98edb60f310 None admin] RBAC: Authorizing `identity:get_project(project_id=0370c388ce4e42d2ad10ebca467e24dc)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:13.154001 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456be928-9f64-463d-8268-e98edb60f310 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:13.160433 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 94/184] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:12 2022] GET /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:13.160433 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456be928-9f64-463d-8268-e98edb60f310 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:13.202817 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4542d41-50d4-4131-904c-78ee2d1d5b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:13.203445 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4542d41-50d4-4131-904c-78ee2d1d5b27 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:13.203722 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4542d41-50d4-4131-904c-78ee2d1d5b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:13.204030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4542d41-50d4-4131-904c-78ee2d1d5b27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:13.214800 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4542d41-50d4-4131-904c-78ee2d1d5b27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:13.219819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-c4542d41-50d4-4131-904c-78ee2d1d5b27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:19:13.219819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:13.219819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:13.219819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:13.219819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:13.219819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:13.219819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:13.219819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:13.219819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:13.219819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:13.219819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:13.219819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:13.219819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:13.219819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:13.219819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:13.219819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:13.220745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:13.220745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:13.220745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:13.220745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:13.220745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:13.220745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:13.220745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:13.220745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:13.220745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:13.220745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:13.220745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:13.220745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:13.220745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:13.220745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:13.220745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:13.220745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:13.221651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:13.221651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:13.221651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:13.221651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:13.221651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:13.221651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:13.221651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:13.221651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:13.221651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:13.221651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:13.221651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:19:13.221651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:13.221651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 91/185] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:19:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:13.223305 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be91cca-1a42-4d65-b187-814d3b9d4fac None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:13.223305 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be91cca-1a42-4d65-b187-814d3b9d4fac None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:13.223305 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be91cca-1a42-4d65-b187-814d3b9d4fac None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:13.224196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 92/186] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:13.224884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be91cca-1a42-4d65-b187-814d3b9d4fac None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:13.287002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456be928-9f64-463d-8268-e98edb60f310 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:13.338958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c513c24-838d-440c-ba49-5fd7fa51210e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:13.339176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:13.339648 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c513c24-838d-440c-ba49-5fd7fa51210e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:13.339980 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c513c24-838d-440c-ba49-5fd7fa51210e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:13.340445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c513c24-838d-440c-ba49-5fd7fa51210e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:13.341314 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c513c24-838d-440c-ba49-5fd7fa51210e None admin] RBAC: Adding query filter params (role.id=89cf5ed93cc642588a54639342c59ea6, scope.project.id=5355bb857aa0482399be52bcee61d585, user.id=0c802f73a7a64d1b862c3589c9d42f5b) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:13.341637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c513c24-838d-440c-ba49-5fd7fa51210e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:13.343145 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c513c24-838d-440c-ba49-5fd7fa51210e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:13.349585 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 95/187] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:19:13 2022] GET /identity/v3/role_assignments?user.id=0c802f73a7a64d1b862c3589c9d42f5b&scope.project.id=5355bb857aa0482399be52bcee61d585&role.id=89cf5ed93cc642588a54639342c59ea6 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:13.350641 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c513c24-838d-440c-ba49-5fd7fa51210e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:13.363731 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be91cca-1a42-4d65-b187-814d3b9d4fac None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:13.379837 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:13.413955 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad7b18-68b5-4ea7-a2e8-f5d6c378b9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:13.414578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad7b18-68b5-4ea7-a2e8-f5d6c378b9a8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:13.414859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad7b18-68b5-4ea7-a2e8-f5d6c378b9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:13.415129 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad7b18-68b5-4ea7-a2e8-f5d6c378b9a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:13.415894 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad7b18-68b5-4ea7-a2e8-f5d6c378b9a8 None admin] RBAC: Adding query filter params (role.id=578c9a17c0fa434f9602f5d2b3648785, scope.project.id=0370c388ce4e42d2ad10ebca467e24dc, user.id=0c802f73a7a64d1b862c3589c9d42f5b) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:13.416241 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad7b18-68b5-4ea7-a2e8-f5d6c378b9a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:13.417787 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad7b18-68b5-4ea7-a2e8-f5d6c378b9a8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:13.424923 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 93/188] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:19:13 2022] GET /identity/v3/role_assignments?user.id=0c802f73a7a64d1b862c3589c9d42f5b&scope.project.id=0370c388ce4e42d2ad10ebca467e24dc&role.id=578c9a17c0fa434f9602f5d2b3648785 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:13.425599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad7b18-68b5-4ea7-a2e8-f5d6c378b9a8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:13.486445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c513c24-838d-440c-ba49-5fd7fa51210e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:13.536095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e8988-c1d1-4601-8241-fc84d6661600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:13.536765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e8988-c1d1-4601-8241-fc84d6661600 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:13.537113 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e8988-c1d1-4601-8241-fc84d6661600 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:13.537330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e8988-c1d1-4601-8241-fc84d6661600 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:13.553303 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e8988-c1d1-4601-8241-fc84d6661600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:13.557065 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad7b18-68b5-4ea7-a2e8-f5d6c378b9a8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:13.561490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-8c9e8988-c1d1-4601-8241-fc84d6661600 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:19:13.561490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:13.561490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:13.561490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:13.561490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:13.561490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:13.561490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:13.561490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:13.561490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:13.561490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:13.561490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:13.561490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:13.561490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:13.561490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:13.561490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:13.561490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:13.562660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:13.562660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:13.562660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:13.562660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:13.562660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:13.562660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:13.562660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:13.562660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:13.562660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:13.562660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:13.562660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:13.562660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:13.562660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:13.562660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:13.562660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:13.562660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:13.563582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:13.563582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:13.563582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:13.563582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:13.563582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:13.563582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:13.563582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:13.563582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:13.563582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:13.563582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:13.563582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:19:13.563582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:13.563582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 96/189] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:13.564861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d8135-c016-49a8-b215-b3cef815e183 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:13.564927 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d8135-c016-49a8-b215-b3cef815e183 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:13.565165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d8135-c016-49a8-b215-b3cef815e183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:13.586238 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d11d8135-c016-49a8-b215-b3cef815e183 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:13.587466 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d11d8135-c016-49a8-b215-b3cef815e183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:13.615217 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8456766-8c80-491e-88df-343ec7c576e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:13.616099 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8456766-8c80-491e-88df-343ec7c576e2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:13.616495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8456766-8c80-491e-88df-343ec7c576e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:13.616901 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8456766-8c80-491e-88df-343ec7c576e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:13.618031 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8456766-8c80-491e-88df-343ec7c576e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:13.618031 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8456766-8c80-491e-88df-343ec7c576e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:13.619618 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8456766-8c80-491e-88df-343ec7c576e2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:13.630267 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 94/190] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:13.633931 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8456766-8c80-491e-88df-343ec7c576e2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:13.652045 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:13.683026 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 97/191] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:13.683740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d8135-c016-49a8-b215-b3cef815e183 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:13.707396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:13.759006 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8456766-8c80-491e-88df-343ec7c576e2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:13.796277 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2158d-d983-4061-8ec0-0fb24fa1decb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:13.796958 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2158d-d983-4061-8ec0-0fb24fa1decb None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:13.797237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2158d-d983-4061-8ec0-0fb24fa1decb None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:13.797510 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2158d-d983-4061-8ec0-0fb24fa1decb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:13.803489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d8135-c016-49a8-b215-b3cef815e183 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:13.807794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb2158d-d983-4061-8ec0-0fb24fa1decb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:13.813176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-adb2158d-d983-4061-8ec0-0fb24fa1decb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:13.813176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:13.813176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:13.813176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:13.813176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:13.813176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:13.813176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:13.813176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:13.813176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:13.813176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:13.813176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:13.813176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:13.813176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:13.813176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:13.813176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:13.813176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:13.814236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:13.814236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:13.814236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:13.814236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:13.814236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:13.814236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:13.814236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:13.814236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:13.814236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:13.814236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:13.814236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:13.814236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:13.814236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:13.814236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:13.814236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:13.814236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:13.815209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:13.815209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:13.815209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:13.815209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:13.815209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:13.815209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:13.815209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:13.815209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:13.815209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:13.815209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:13.815209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:13.815209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:13.815209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 95/192] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:19:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:13.815209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2158d-d983-4061-8ec0-0fb24fa1decb None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:13.851195 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28868d2f-cdff-4af3-87d5-df3b5becd09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:13.852001 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28868d2f-cdff-4af3-87d5-df3b5becd09c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:13.852383 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28868d2f-cdff-4af3-87d5-df3b5becd09c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:13.852759 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28868d2f-cdff-4af3-87d5-df3b5becd09c None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:13.853535 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28868d2f-cdff-4af3-87d5-df3b5becd09c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:13.853971 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28868d2f-cdff-4af3-87d5-df3b5becd09c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:13.855791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28868d2f-cdff-4af3-87d5-df3b5becd09c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:13.865632 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 98/193] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:13.867107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28868d2f-cdff-4af3-87d5-df3b5becd09c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:13.890370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:13.924806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2158d-d983-4061-8ec0-0fb24fa1decb None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:13.951868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:13.961240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc1dbd8-cacb-4c90-a94d-ac18a15a59f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:13.961873 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1dbd8-cacb-4c90-a94d-ac18a15a59f4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:13.962148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1dbd8-cacb-4c90-a94d-ac18a15a59f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:13.962422 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1dbd8-cacb-4c90-a94d-ac18a15a59f4 None admin] PATH_INFO: `/v3/users/5f2be18779f74c668807d3743358c46c` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:13.976531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc1dbd8-cacb-4c90-a94d-ac18a15a59f4 None admin] RBAC: Authorizing `identity:get_user(user_id=5f2be18779f74c668807d3743358c46c)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:13.976720 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28868d2f-cdff-4af3-87d5-df3b5becd09c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:13.977984 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc1dbd8-cacb-4c90-a94d-ac18a15a59f4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:13.979630 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 96/194] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:13 2022] GET /identity/v3/users/5f2be18779f74c668807d3743358c46c => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:13.980416 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc1dbd8-cacb-4c90-a94d-ac18a15a59f4 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:14.013522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172e958-3146-406e-8dbd-97a303391dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:14.014148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172e958-3146-406e-8dbd-97a303391dad None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:14.014425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172e958-3146-406e-8dbd-97a303391dad None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:14.014695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172e958-3146-406e-8dbd-97a303391dad None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:14.015368 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172e958-3146-406e-8dbd-97a303391dad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:14.015666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172e958-3146-406e-8dbd-97a303391dad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:14.017075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172e958-3146-406e-8dbd-97a303391dad None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:14.030995 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 99/195] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:19:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:14.030995 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172e958-3146-406e-8dbd-97a303391dad None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:14.085981 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc1dbd8-cacb-4c90-a94d-ac18a15a59f4 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:14.116150 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:14.122431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ecffe-6168-4454-9b5e-0240c95d513a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:14.123126 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ecffe-6168-4454-9b5e-0240c95d513a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:14.123461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ecffe-6168-4454-9b5e-0240c95d513a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:14.123803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ecffe-6168-4454-9b5e-0240c95d513a None admin] PATH_INFO: `/v3/users/8bd7790fcaab414d8cd6cab1d09660a1` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:14.125131 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ecffe-6168-4454-9b5e-0240c95d513a None admin] RBAC: Authorizing `identity:get_user(user_id=8bd7790fcaab414d8cd6cab1d09660a1)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:14.126621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ecffe-6168-4454-9b5e-0240c95d513a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:14.128260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 97/196] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:13 2022] GET /identity/v3/users/8bd7790fcaab414d8cd6cab1d09660a1 => generated 289 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:14.133160 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172e958-3146-406e-8dbd-97a303391dad None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:14.133860 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ecffe-6168-4454-9b5e-0240c95d513a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:14.167822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12494e94-8cc0-4bca-8c78-964923234d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:14.168488 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12494e94-8cc0-4bca-8c78-964923234d01 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:14.168763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12494e94-8cc0-4bca-8c78-964923234d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:14.169040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12494e94-8cc0-4bca-8c78-964923234d01 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:14.170162 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12494e94-8cc0-4bca-8c78-964923234d01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:14.171618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12494e94-8cc0-4bca-8c78-964923234d01 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:14.173216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 100/197] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:19:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:14.175507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12494e94-8cc0-4bca-8c78-964923234d01 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:14.185702 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:14.243956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ecffe-6168-4454-9b5e-0240c95d513a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:14.279420 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14be57-0f7e-441e-939f-9f40311f7b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:14.280057 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14be57-0f7e-441e-939f-9f40311f7b49 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:14.280337 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14be57-0f7e-441e-939f-9f40311f7b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:14.280599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14be57-0f7e-441e-939f-9f40311f7b49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:14.287285 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12494e94-8cc0-4bca-8c78-964923234d01 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:14.322989 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd217f-0e58-49b4-9d56-0bf253344784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:14.323629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd217f-0e58-49b4-9d56-0bf253344784 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:14.323909 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd217f-0e58-49b4-9d56-0bf253344784 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:14.324213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd217f-0e58-49b4-9d56-0bf253344784 None admin] PATH_INFO: `/v3/users/5f2be18779f74c668807d3743358c46c` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:14.325324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd217f-0e58-49b4-9d56-0bf253344784 None admin] RBAC: Authorizing `identity:get_user(user_id=5f2be18779f74c668807d3743358c46c)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:14.326798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd217f-0e58-49b4-9d56-0bf253344784 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:14.328396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 101/198] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:14 2022] GET /identity/v3/users/5f2be18779f74c668807d3743358c46c => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:14.329163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd217f-0e58-49b4-9d56-0bf253344784 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:14.364485 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:14.368643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b14be57-0f7e-441e-939f-9f40311f7b49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:14.370241 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b14be57-0f7e-441e-939f-9f40311f7b49 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:14.412607 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:14.438199 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd217f-0e58-49b4-9d56-0bf253344784 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:14.473184 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1b913-9f17-46a3-84d9-ff740439acdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:14.473809 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1b913-9f17-46a3-84d9-ff740439acdd None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:14.474087 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1b913-9f17-46a3-84d9-ff740439acdd None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:14.474559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1b913-9f17-46a3-84d9-ff740439acdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:14.478639 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 98/199] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:14.479545 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14be57-0f7e-441e-939f-9f40311f7b49 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:14.545230 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1b913-9f17-46a3-84d9-ff740439acdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:14.546686 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1b913-9f17-46a3-84d9-ff740439acdd None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:14.575394 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:14.578902 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14be57-0f7e-441e-939f-9f40311f7b49 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:14.614337 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891b44f-c0f5-4f66-8a06-1be06295b297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:14.615000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891b44f-c0f5-4f66-8a06-1be06295b297 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:14.615225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891b44f-c0f5-4f66-8a06-1be06295b297 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:14.615529 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4891b44f-c0f5-4f66-8a06-1be06295b297 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:14.616198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4891b44f-c0f5-4f66-8a06-1be06295b297 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:14.617697 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4891b44f-c0f5-4f66-8a06-1be06295b297 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:14.639564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:14.655976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 99/200] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 29 20:19:14 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:19:14.655976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891b44f-c0f5-4f66-8a06-1be06295b297 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:14.673253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 102/201] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:14.674299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1b913-9f17-46a3-84d9-ff740439acdd None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:14.777347 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4891b44f-c0f5-4f66-8a06-1be06295b297 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:14.801314 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1b913-9f17-46a3-84d9-ff740439acdd None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:14.815111 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed3c21-18d8-40bb-9887-b60108186a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:14.815733 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed3c21-18d8-40bb-9887-b60108186a63 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:14.816044 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed3c21-18d8-40bb-9887-b60108186a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:14.816317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed3c21-18d8-40bb-9887-b60108186a63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:14.842609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:14.855010 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a5e56-ba0b-4e3f-adbe-902fb773a3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:14.855659 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a5e56-ba0b-4e3f-adbe-902fb773a3ae None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:14.855973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a5e56-ba0b-4e3f-adbe-902fb773a3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:14.856245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15a5e56-ba0b-4e3f-adbe-902fb773a3ae None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:14.857761 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a5e56-ba0b-4e3f-adbe-902fb773a3ae None admin] RBAC: Authorizing `identity:get_project(project_id=eddee91f1cf44ccf83ca7cd9419ab306)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:14.858812 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15a5e56-ba0b-4e3f-adbe-902fb773a3ae None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:14.860492 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 103/202] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:14 2022] GET /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:14.863106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a5e56-ba0b-4e3f-adbe-902fb773a3ae None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:14.908626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed3c21-18d8-40bb-9887-b60108186a63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:14.910044 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed3c21-18d8-40bb-9887-b60108186a63 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:14.933292 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:15.020881 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15a5e56-ba0b-4e3f-adbe-902fb773a3ae None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:15.053210 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 100/203] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:15.054625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed3c21-18d8-40bb-9887-b60108186a63 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:15.073384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730ef84-81ef-4c93-87cf-e33fd8ef1770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:15.074404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730ef84-81ef-4c93-87cf-e33fd8ef1770 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:15.074715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730ef84-81ef-4c93-87cf-e33fd8ef1770 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:15.075051 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730ef84-81ef-4c93-87cf-e33fd8ef1770 None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:15.077361 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730ef84-81ef-4c93-87cf-e33fd8ef1770 None admin] RBAC: Authorizing `identity:get_project(project_id=0370c388ce4e42d2ad10ebca467e24dc)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:15.079900 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730ef84-81ef-4c93-87cf-e33fd8ef1770 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:15.082147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 104/204] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:14 2022] GET /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:15.087960 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730ef84-81ef-4c93-87cf-e33fd8ef1770 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:15.126667 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:15.181122 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed3c21-18d8-40bb-9887-b60108186a63 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:15.218884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd90d2-0e3c-4cbc-ab4c-cc5204750395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:15.219810 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd90d2-0e3c-4cbc-ab4c-cc5204750395 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:15.220297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd90d2-0e3c-4cbc-ab4c-cc5204750395 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:15.220747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd90d2-0e3c-4cbc-ab4c-cc5204750395 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:15.221638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd90d2-0e3c-4cbc-ab4c-cc5204750395 None admin] RBAC: Adding query filter params (role.id=322da7bd64024770a64bdb73e5c54719, scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306, user.id=5f2be18779f74c668807d3743358c46c) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:15.222109 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd90d2-0e3c-4cbc-ab4c-cc5204750395 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:15.223743 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd90d2-0e3c-4cbc-ab4c-cc5204750395 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:15.230413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 101/205] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:19:15 2022] GET /identity/v3/role_assignments?user.id=5f2be18779f74c668807d3743358c46c&scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306&role.id=322da7bd64024770a64bdb73e5c54719 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:15.231472 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd90d2-0e3c-4cbc-ab4c-cc5204750395 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:15.232919 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730ef84-81ef-4c93-87cf-e33fd8ef1770 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:15.247850 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:15.295434 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6e112-5ccc-47a0-9ba7-85f3c11bb9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:15.296589 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6e112-5ccc-47a0-9ba7-85f3c11bb9b0 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:15.297062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6e112-5ccc-47a0-9ba7-85f3c11bb9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:15.297552 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6e112-5ccc-47a0-9ba7-85f3c11bb9b0 None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:15.300940 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e6e112-5ccc-47a0-9ba7-85f3c11bb9b0 None admin] RBAC: Authorizing `identity:get_project(project_id=eddee91f1cf44ccf83ca7cd9419ab306)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:15.302648 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e6e112-5ccc-47a0-9ba7-85f3c11bb9b0 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:15.305482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 105/206] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:15 2022] GET /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:15.313843 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6e112-5ccc-47a0-9ba7-85f3c11bb9b0 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:15.455428 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd90d2-0e3c-4cbc-ab4c-cc5204750395 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:15.491468 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:15.519841 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d74ce-a95d-4441-bbd0-738e6f41a6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:15.520984 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d74ce-a95d-4441-bbd0-738e6f41a6f3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:15.521689 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d74ce-a95d-4441-bbd0-738e6f41a6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:15.523585 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d74ce-a95d-4441-bbd0-738e6f41a6f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:15.524726 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d74ce-a95d-4441-bbd0-738e6f41a6f3 None admin] RBAC: Adding query filter params (role.id=5ae3132138594bb7a1fbacc9d93198c1, scope.project.id=0370c388ce4e42d2ad10ebca467e24dc, user.id=8bd7790fcaab414d8cd6cab1d09660a1) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:15.525450 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d74ce-a95d-4441-bbd0-738e6f41a6f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:15.528927 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d74ce-a95d-4441-bbd0-738e6f41a6f3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:15.530927 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6e112-5ccc-47a0-9ba7-85f3c11bb9b0 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:15.544548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 102/207] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:19:15 2022] GET /identity/v3/role_assignments?user.id=8bd7790fcaab414d8cd6cab1d09660a1&scope.project.id=0370c388ce4e42d2ad10ebca467e24dc&role.id=5ae3132138594bb7a1fbacc9d93198c1 => generated 608 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:15.575639 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa723960-9ddc-4a53-a8a5-574c3f6dcdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:15.576697 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa723960-9ddc-4a53-a8a5-574c3f6dcdb3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:15.577172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa723960-9ddc-4a53-a8a5-574c3f6dcdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:15.577669 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa723960-9ddc-4a53-a8a5-574c3f6dcdb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:15.578679 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa723960-9ddc-4a53-a8a5-574c3f6dcdb3 None admin] RBAC: Adding query filter params (role.id=578c9a17c0fa434f9602f5d2b3648785, scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306, user.id=5f2be18779f74c668807d3743358c46c) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:15.579181 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa723960-9ddc-4a53-a8a5-574c3f6dcdb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:15.580932 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa723960-9ddc-4a53-a8a5-574c3f6dcdb3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:15.588079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 106/208] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:19:15 2022] GET /identity/v3/role_assignments?user.id=5f2be18779f74c668807d3743358c46c&scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306&role.id=578c9a17c0fa434f9602f5d2b3648785 => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:16.984918 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9d471-aed7-4108-8cfc-4c7fbb8a0657 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:16.986660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9d471-aed7-4108-8cfc-4c7fbb8a0657 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:16.987427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e9d471-aed7-4108-8cfc-4c7fbb8a0657 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:16.990226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 103/209] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:16.996493 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7f614-f594-4946-8f6c-379515cfae7a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:16.997174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7f614-f594-4946-8f6c-379515cfae7a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:16.997174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7f614-f594-4946-8f6c-379515cfae7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:17.015459 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-5ba7f614-f594-4946-8f6c-379515cfae7a None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:17.016689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5ba7f614-f594-4946-8f6c-379515cfae7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:17.061128 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:17.102882 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 107/210] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:19:17.196244 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc4d25-1903-4f99-a3a8-51d575218181 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:17.196527 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc4d25-1903-4f99-a3a8-51d575218181 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:17.196801 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc4d25-1903-4f99-a3a8-51d575218181 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:17.198468 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 104/211] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:17.203817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e0f3e-1eec-4d7e-9d56-b5642722eaef None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:17.204133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e0f3e-1eec-4d7e-9d56-b5642722eaef None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:17.204412 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e0f3e-1eec-4d7e-9d56-b5642722eaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:17.222603 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-af0e0f3e-1eec-4d7e-9d56-b5642722eaef None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:17.223791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-af0e0f3e-1eec-4d7e-9d56-b5642722eaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:17.290607 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2525e50-8655-4cf8-a373-b44e877f8ac3 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:17.291228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2525e50-8655-4cf8-a373-b44e877f8ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:17.291701 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2525e50-8655-4cf8-a373-b44e877f8ac3 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:17.293287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 105/212] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:19:17.298055 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 108/213] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:17.299019 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bfda0-f3cb-462b-8e01-0709a1e1c573 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:17.299518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bfda0-f3cb-462b-8e01-0709a1e1c573 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:17.300020 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bfda0-f3cb-462b-8e01-0709a1e1c573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:17.319873 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6e5bfda0-f3cb-462b-8e01-0709a1e1c573 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:17.321968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6e5bfda0-f3cb-462b-8e01-0709a1e1c573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:17.378268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e0f3e-1eec-4d7e-9d56-b5642722eaef None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:17.394409 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 106/214] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:19:17.484153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e0f3e-1eec-4d7e-9d56-b5642722eaef None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:17.485688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:17.519740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df174c-cbc2-4fc4-82dc-a3a7034a0bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:17.520418 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df174c-cbc2-4fc4-82dc-a3a7034a0bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:17.520706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df174c-cbc2-4fc4-82dc-a3a7034a0bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:17.520982 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35df174c-cbc2-4fc4-82dc-a3a7034a0bc1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:17.522089 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df174c-cbc2-4fc4-82dc-a3a7034a0bc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:17.523577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35df174c-cbc2-4fc4-82dc-a3a7034a0bc1 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:17.525487 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 109/215] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:19:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:17.531082 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bfda0-f3cb-462b-8e01-0709a1e1c573 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:17.594788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df174c-cbc2-4fc4-82dc-a3a7034a0bc1 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:17.643103 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bfda0-f3cb-462b-8e01-0709a1e1c573 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:17.660919 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:17.683621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361c99d-90d5-44e4-b7b8-b819ed054173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:17.684514 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361c99d-90d5-44e4-b7b8-b819ed054173 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:17.684789 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361c99d-90d5-44e4-b7b8-b819ed054173 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:17.685058 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361c99d-90d5-44e4-b7b8-b819ed054173 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:17.685720 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361c99d-90d5-44e4-b7b8-b819ed054173 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:17.687166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361c99d-90d5-44e4-b7b8-b819ed054173 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:17.711712 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35df174c-cbc2-4fc4-82dc-a3a7034a0bc1 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:17.712515 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 107/216] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 29 20:19:17 2022] POST /identity/v3/users => generated 297 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:19:17.713790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361c99d-90d5-44e4-b7b8-b819ed054173 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:17.755116 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0533a68-50fd-4130-981f-1ff038323e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:17.755888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0533a68-50fd-4130-981f-1ff038323e7e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:17.756250 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0533a68-50fd-4130-981f-1ff038323e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:17.756616 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0533a68-50fd-4130-981f-1ff038323e7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:17.770734 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0533a68-50fd-4130-981f-1ff038323e7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:17.776817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-d0533a68-50fd-4130-981f-1ff038323e7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:17.776817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:17.776817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:17.776817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:17.776817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:17.776817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:17.776817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:17.776817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:17.776817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:17.776817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:17.776817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:17.776817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:17.776817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:17.776817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:17.776817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:17.776817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:17.780288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:17.780288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:17.780288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:17.780288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:17.780288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:17.780288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:17.780288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:17.780288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:17.780288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:17.780288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:17.780288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:17.780288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:17.780288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:17.780288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:17.780288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:17.780288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:17.781141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:17.781141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:17.781141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:17.781141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:17.781141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:17.781141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:17.781141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:17.781141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:17.781141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:17.781141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:17.781141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:17.781141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:17.781141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 110/217] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:19:17.786123 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0533a68-50fd-4130-981f-1ff038323e7e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:17.868431 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:17.926710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361c99d-90d5-44e4-b7b8-b819ed054173 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:17.964354 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4ccf8-e9a8-404d-acab-212f2b24f8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:17.965193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4ccf8-e9a8-404d-acab-212f2b24f8d5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:17.965619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4ccf8-e9a8-404d-acab-212f2b24f8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:17.965957 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4ccf8-e9a8-404d-acab-212f2b24f8d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:17.967325 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0533a68-50fd-4130-981f-1ff038323e7e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:17.978070 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4ccf8-e9a8-404d-acab-212f2b24f8d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:17.983610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-72d4ccf8-e9a8-404d-acab-212f2b24f8d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:19:17.983610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:17.983610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:17.983610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:17.983610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:17.983610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:17.983610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:17.983610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:17.983610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:17.983610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:17.983610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:17.983610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:17.983610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:17.983610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:17.983610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:17.983610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:17.984563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:17.984563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:17.984563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:17.984563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:17.984563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:17.984563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:17.984563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:17.984563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:17.984563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:17.984563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:17.984563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:17.984563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:17.984563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:17.984563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:17.984563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:17.984563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:17.985772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:17.985772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:17.985772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:17.985772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:17.985772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:17.985772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:17.985772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:17.985772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:17.985772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:17.985772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:17.985772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:19:17.985772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:17.990651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 108/218] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:19:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:17.991627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4ccf8-e9a8-404d-acab-212f2b24f8d5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:18.012438 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca13b8e-0fa5-4f9c-9d66-e80cd60be110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:18.013793 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca13b8e-0fa5-4f9c-9d66-e80cd60be110 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:18.014821 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca13b8e-0fa5-4f9c-9d66-e80cd60be110 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:18.015315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca13b8e-0fa5-4f9c-9d66-e80cd60be110 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:18.016258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca13b8e-0fa5-4f9c-9d66-e80cd60be110 None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:18.016814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca13b8e-0fa5-4f9c-9d66-e80cd60be110 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:18.018521 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca13b8e-0fa5-4f9c-9d66-e80cd60be110 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:18.026743 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:18.035642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 111/219] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:18.038572 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca13b8e-0fa5-4f9c-9d66-e80cd60be110 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:18.152759 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4ccf8-e9a8-404d-acab-212f2b24f8d5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:18.204712 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19e8ce-16ae-498b-99cc-a1f57a5b9e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:18.205498 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19e8ce-16ae-498b-99cc-a1f57a5b9e75 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:18.205926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19e8ce-16ae-498b-99cc-a1f57a5b9e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:18.206297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19e8ce-16ae-498b-99cc-a1f57a5b9e75 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:18.207067 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a19e8ce-16ae-498b-99cc-a1f57a5b9e75 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:18.207489 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a19e8ce-16ae-498b-99cc-a1f57a5b9e75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:18.209021 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a19e8ce-16ae-498b-99cc-a1f57a5b9e75 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:18.216140 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:18.223347 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 109/220] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:18.229213 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35e93d-2139-4dda-9ade-7eafcb352fc0 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:18.229502 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35e93d-2139-4dda-9ade-7eafcb352fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:18.229781 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35e93d-2139-4dda-9ade-7eafcb352fc0 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:18.231142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 110/221] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:18.231879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35e93d-2139-4dda-9ade-7eafcb352fc0 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:18.235725 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca13b8e-0fa5-4f9c-9d66-e80cd60be110 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:18.284496 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d1afc-68c2-4c9d-9a47-3a7dc4cb8767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:18.285159 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d1afc-68c2-4c9d-9a47-3a7dc4cb8767 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:18.285496 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d1afc-68c2-4c9d-9a47-3a7dc4cb8767 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:18.285773 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d1afc-68c2-4c9d-9a47-3a7dc4cb8767 None admin] PATH_INFO: `/v3/users/01e34eec23f243ebafc5a3257a700356` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:18.305069 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d1afc-68c2-4c9d-9a47-3a7dc4cb8767 None admin] RBAC: Authorizing `identity:get_user(user_id=01e34eec23f243ebafc5a3257a700356)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:18.312994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d1afc-68c2-4c9d-9a47-3a7dc4cb8767 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:18.315642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 112/222] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:18 2022] GET /identity/v3/users/01e34eec23f243ebafc5a3257a700356 => generated 297 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:18.317018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e71ebe-72d5-4734-a9f4-c695e354f9a3 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:18.317365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e71ebe-72d5-4734-a9f4-c695e354f9a3 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:18.317647 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e71ebe-72d5-4734-a9f4-c695e354f9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:18.363692 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-70e71ebe-72d5-4734-a9f4-c695e354f9a3 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:18.365198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-70e71ebe-72d5-4734-a9f4-c695e354f9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:18.395349 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:18.418537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35e93d-2139-4dda-9ade-7eafcb352fc0 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:18.462683 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c5b80-c73f-424a-b677-9828ac98aa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:18.464478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c5b80-c73f-424a-b677-9828ac98aa2a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:18.465250 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c5b80-c73f-424a-b677-9828ac98aa2a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:18.465743 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c5b80-c73f-424a-b677-9828ac98aa2a None admin] PATH_INFO: `/v3/users/5f2be18779f74c668807d3743358c46c` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:18.468276 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c5b80-c73f-424a-b677-9828ac98aa2a None admin] RBAC: Authorizing `identity:get_user(user_id=5f2be18779f74c668807d3743358c46c)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:18.470171 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c5b80-c73f-424a-b677-9828ac98aa2a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:18.472055 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 111/223] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:18 2022] GET /identity/v3/users/5f2be18779f74c668807d3743358c46c => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:18.472829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 113/224] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:18.473023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c5b80-c73f-424a-b677-9828ac98aa2a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:18.483983 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e71ebe-72d5-4734-a9f4-c695e354f9a3 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:18.584539 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:18.595805 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c5b80-c73f-424a-b677-9828ac98aa2a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:18.623731 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e71ebe-72d5-4734-a9f4-c695e354f9a3 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:18.632550 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e48a6-d60e-44a5-b599-c82182681376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:18.633204 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e48a6-d60e-44a5-b599-c82182681376 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:18.633524 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e48a6-d60e-44a5-b599-c82182681376 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:18.633798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e48a6-d60e-44a5-b599-c82182681376 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:18.664894 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:18.701300 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86fa5c8-1105-4e05-a007-31aba118ccf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:18.701874 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fa5c8-1105-4e05-a007-31aba118ccf3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:18.702162 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fa5c8-1105-4e05-a007-31aba118ccf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:18.702441 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fa5c8-1105-4e05-a007-31aba118ccf3 None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:18.707025 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e48a6-d60e-44a5-b599-c82182681376 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:18.708563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e48a6-d60e-44a5-b599-c82182681376 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:18.708679 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86fa5c8-1105-4e05-a007-31aba118ccf3 None admin] RBAC: Authorizing `identity:get_project(project_id=eddee91f1cf44ccf83ca7cd9419ab306)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:18.710208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86fa5c8-1105-4e05-a007-31aba118ccf3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:18.713066 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 114/225] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:18 2022] GET /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:18.721609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86fa5c8-1105-4e05-a007-31aba118ccf3 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:18.897512 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86fa5c8-1105-4e05-a007-31aba118ccf3 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:18.903302 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 112/226] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:18.904198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e48a6-d60e-44a5-b599-c82182681376 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:18.951448 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61ea65-9503-43f5-a12a-072c637bf45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:18.952078 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61ea65-9503-43f5-a12a-072c637bf45a None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:18.952363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61ea65-9503-43f5-a12a-072c637bf45a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:18.952638 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61ea65-9503-43f5-a12a-072c637bf45a None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306/users/5f2be18779f74c668807d3743358c46c/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:18.954623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61ea65-9503-43f5-a12a-072c637bf45a None admin] RBAC: Authorizing `identity:create_grant(project_id=eddee91f1cf44ccf83ca7cd9419ab306, user_id=5f2be18779f74c668807d3743358c46c, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:18.956646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61ea65-9503-43f5-a12a-072c637bf45a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:18.974930 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 115/227] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:19:18 2022] PUT /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306/users/5f2be18779f74c668807d3743358c46c/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:19:18.975781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61ea65-9503-43f5-a12a-072c637bf45a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:18.998214 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:19.009841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:19.085033 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e48a6-d60e-44a5-b599-c82182681376 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:19.160766 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613aadd7-602b-41e1-87f2-595b34f43091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:19.161804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aadd7-602b-41e1-87f2-595b34f43091 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:19.162323 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aadd7-602b-41e1-87f2-595b34f43091 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:19.162595 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613aadd7-602b-41e1-87f2-595b34f43091 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:19.173497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61ea65-9503-43f5-a12a-072c637bf45a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:19.188411 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613aadd7-602b-41e1-87f2-595b34f43091 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:19.193423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-613aadd7-602b-41e1-87f2-595b34f43091 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:19.193423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:19.193423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:19.193423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:19.193423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:19.193423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:19.193423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:19.193423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:19.193423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:19.193423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:19.193423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:19.193423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:19.193423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:19.193423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:19.193423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:19.193423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:19.196663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:19.196663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:19.196663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:19.196663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:19.196663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:19.196663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:19.196663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:19.196663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:19.196663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:19.196663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:19.196663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:19.196663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:19.196663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:19.196663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:19.196663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:19.196663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:19.198870 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:19.198870 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:19.198870 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:19.198870 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:19.198870 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:19.198870 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:19.198870 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:19.198870 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:19.198870 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:19.198870 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:19.198870 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:19.198870 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:19.198870 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 113/228] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:19:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:19.209164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613aadd7-602b-41e1-87f2-595b34f43091 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:19.274148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea71b45-367c-4882-82ac-f4ae5e2ce3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:19.275078 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea71b45-367c-4882-82ac-f4ae5e2ce3c3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:19.275078 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea71b45-367c-4882-82ac-f4ae5e2ce3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:19.275344 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea71b45-367c-4882-82ac-f4ae5e2ce3c3 None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:19.280798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea71b45-367c-4882-82ac-f4ae5e2ce3c3 None admin] RBAC: Authorizing `identity:get_project(project_id=eddee91f1cf44ccf83ca7cd9419ab306)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:19.282685 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea71b45-367c-4882-82ac-f4ae5e2ce3c3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:19.284479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 116/229] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:18 2022] GET /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306 => generated 302 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:19.296028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea71b45-367c-4882-82ac-f4ae5e2ce3c3 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:19.332658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613aadd7-602b-41e1-87f2-595b34f43091 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:19.352281 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:19.373235 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba57af9-2bfd-4a6c-9c5f-31351b650c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:19.373994 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba57af9-2bfd-4a6c-9c5f-31351b650c13 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:19.374277 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba57af9-2bfd-4a6c-9c5f-31351b650c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:19.374548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba57af9-2bfd-4a6c-9c5f-31351b650c13 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:19.375222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba57af9-2bfd-4a6c-9c5f-31351b650c13 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:19.375529 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba57af9-2bfd-4a6c-9c5f-31351b650c13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:19.377274 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba57af9-2bfd-4a6c-9c5f-31351b650c13 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:19.387203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 114/230] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:19:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:19.392782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba57af9-2bfd-4a6c-9c5f-31351b650c13 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:19.505108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba57af9-2bfd-4a6c-9c5f-31351b650c13 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:19.541810 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd41462-5ff9-47a9-991a-565baa237c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:19.542455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd41462-5ff9-47a9-991a-565baa237c00 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:19.542728 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd41462-5ff9-47a9-991a-565baa237c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:19.543003 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd41462-5ff9-47a9-991a-565baa237c00 None admin] PATH_INFO: `/v3/users/5f2be18779f74c668807d3743358c46c` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:19.544192 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd41462-5ff9-47a9-991a-565baa237c00 None admin] RBAC: Authorizing `identity:get_user(user_id=5f2be18779f74c668807d3743358c46c)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:19.545831 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd41462-5ff9-47a9-991a-565baa237c00 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:19.547432 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 115/231] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:19 2022] GET /identity/v3/users/5f2be18779f74c668807d3743358c46c => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:19.553864 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd41462-5ff9-47a9-991a-565baa237c00 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:19.566006 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:19.624323 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:19.650284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea71b45-367c-4882-82ac-f4ae5e2ce3c3 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:19.671732 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd41462-5ff9-47a9-991a-565baa237c00 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:19.713848 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c2c6b-1378-48ad-b3a5-3666194c039e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:19.714491 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c2c6b-1378-48ad-b3a5-3666194c039e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:19.714767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c2c6b-1378-48ad-b3a5-3666194c039e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:19.715036 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c2c6b-1378-48ad-b3a5-3666194c039e None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:19.716211 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c2c6b-1378-48ad-b3a5-3666194c039e None admin] RBAC: Authorizing `identity:get_project(project_id=eddee91f1cf44ccf83ca7cd9419ab306)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:19.717838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c2c6b-1378-48ad-b3a5-3666194c039e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:19.719493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 116/232] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:19 2022] GET /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:19.725036 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c2c6b-1378-48ad-b3a5-3666194c039e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:19.801275 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e606199-fa74-470b-98cc-cf37b270bd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:19.801911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e606199-fa74-470b-98cc-cf37b270bd7f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:19.802193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e606199-fa74-470b-98cc-cf37b270bd7f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:19.802466 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e606199-fa74-470b-98cc-cf37b270bd7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:19.803220 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e606199-fa74-470b-98cc-cf37b270bd7f None admin] RBAC: Adding query filter params (role.id=89cf5ed93cc642588a54639342c59ea6, scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306, user.id=01e34eec23f243ebafc5a3257a700356) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:19.803530 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e606199-fa74-470b-98cc-cf37b270bd7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:19.805065 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e606199-fa74-470b-98cc-cf37b270bd7f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:19.826141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 117/233] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:19:19 2022] GET /identity/v3/role_assignments?user.id=01e34eec23f243ebafc5a3257a700356&scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306&role.id=89cf5ed93cc642588a54639342c59ea6 => generated 259 bytes in 531 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:19.833425 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c2c6b-1378-48ad-b3a5-3666194c039e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:19.872042 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9af8d-2b0d-4a57-938a-1d6a426747fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:19.872777 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9af8d-2b0d-4a57-938a-1d6a426747fe None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:19.873054 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9af8d-2b0d-4a57-938a-1d6a426747fe None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:19.873321 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9af8d-2b0d-4a57-938a-1d6a426747fe None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306/users/5f2be18779f74c668807d3743358c46c/roles/578c9a17c0fa434f9602f5d2b3648785` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:19.875595 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9af8d-2b0d-4a57-938a-1d6a426747fe None admin] RBAC: Authorizing `identity:create_grant(project_id=eddee91f1cf44ccf83ca7cd9419ab306, user_id=5f2be18779f74c668807d3743358c46c, role_id=578c9a17c0fa434f9602f5d2b3648785)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:19.877085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9af8d-2b0d-4a57-938a-1d6a426747fe None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:19.884589 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:19.894635 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 117/234] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:19:19 2022] PUT /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306/users/5f2be18779f74c668807d3743358c46c/roles/578c9a17c0fa434f9602f5d2b3648785 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:19:20.269953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e9c1b-cb37-43ab-83bc-c49e8cf96d72 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:20.270630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e9c1b-cb37-43ab-83bc-c49e8cf96d72 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:20.270630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e9c1b-cb37-43ab-83bc-c49e8cf96d72 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:20.271909 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 118/235] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:20.287118 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06dcdc-60c7-4e5e-bf95-1bdacbee4e13 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:20.287716 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06dcdc-60c7-4e5e-bf95-1bdacbee4e13 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:20.287716 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06dcdc-60c7-4e5e-bf95-1bdacbee4e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:20.314084 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-4d06dcdc-60c7-4e5e-bf95-1bdacbee4e13 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:20.315285 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4d06dcdc-60c7-4e5e-bf95-1bdacbee4e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:20.335851 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b7780-b83a-4a18-89be-1332ad011d81 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:20.336220 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b7780-b83a-4a18-89be-1332ad011d81 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:20.336471 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b7780-b83a-4a18-89be-1332ad011d81 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:20.337963 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 119/236] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:19:20.357524 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfac5fa-f950-4978-942c-f097c4dc67cb None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:20.357524 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfac5fa-f950-4978-942c-f097c4dc67cb None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:20.357524 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfac5fa-f950-4978-942c-f097c4dc67cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:20.381029 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4bfac5fa-f950-4978-942c-f097c4dc67cb None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:20.387987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4bfac5fa-f950-4978-942c-f097c4dc67cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:20.433561 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 118/237] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:20.477985 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 120/238] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:20.733392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06dcdc-60c7-4e5e-bf95-1bdacbee4e13 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:20.811342 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfac5fa-f950-4978-942c-f097c4dc67cb None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:20.900523 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06dcdc-60c7-4e5e-bf95-1bdacbee4e13 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:20.911074 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:20.932370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:20.965696 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87898bb-ab47-4275-b069-bb20ac18dc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:20.966677 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87898bb-ab47-4275-b069-bb20ac18dc67 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:20.967169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87898bb-ab47-4275-b069-bb20ac18dc67 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:20.968060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87898bb-ab47-4275-b069-bb20ac18dc67 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:20.973148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfac5fa-f950-4978-942c-f097c4dc67cb None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:21.000973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87898bb-ab47-4275-b069-bb20ac18dc67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:21.006461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-b87898bb-ab47-4275-b069-bb20ac18dc67 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:19:21.006461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:21.006461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:21.006461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:21.006461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:21.006461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:21.006461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:21.006461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:21.006461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:21.006461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:21.006461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:21.006461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:21.006461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:21.006461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:21.006461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:21.006461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:21.007359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:21.007359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:21.007359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:21.007359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:21.007359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:21.007359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:21.007359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:21.007359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:21.007359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:21.007359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:21.007359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:21.007359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:21.007359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:21.007359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:21.007359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:21.007359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:21.008382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:21.008382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:21.008382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:21.008382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:21.008382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:21.008382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:21.008382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:21.008382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:21.008382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:21.008382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:21.008382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:19:21.008382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:21.013373 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 119/239] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:21.016536 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de48481-f335-4da0-9563-809bd2b8ac81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:21.018192 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87898bb-ab47-4275-b069-bb20ac18dc67 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:21.019320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de48481-f335-4da0-9563-809bd2b8ac81 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:21.020467 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de48481-f335-4da0-9563-809bd2b8ac81 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:21.020755 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de48481-f335-4da0-9563-809bd2b8ac81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:21.023549 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de48481-f335-4da0-9563-809bd2b8ac81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:21.025818 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de48481-f335-4da0-9563-809bd2b8ac81 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:21.028516 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 121/240] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:19:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:21.034373 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de48481-f335-4da0-9563-809bd2b8ac81 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:21.189304 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de48481-f335-4da0-9563-809bd2b8ac81 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:21.226416 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87898bb-ab47-4275-b069-bb20ac18dc67 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:21.233359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260bcae-8a58-4ef7-8599-8705cb063d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:21.234250 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260bcae-8a58-4ef7-8599-8705cb063d5b None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:21.234794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260bcae-8a58-4ef7-8599-8705cb063d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:21.235256 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260bcae-8a58-4ef7-8599-8705cb063d5b None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:21.235927 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8260bcae-8a58-4ef7-8599-8705cb063d5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:21.237389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8260bcae-8a58-4ef7-8599-8705cb063d5b None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:21.249758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:21.265609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 122/241] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 29 20:19:21 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:19:21.280925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f3d5b-4f8f-4395-8fa2-2e7dae5ad256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:21.281629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f3d5b-4f8f-4395-8fa2-2e7dae5ad256 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:21.281967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f3d5b-4f8f-4395-8fa2-2e7dae5ad256 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:21.282248 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f3d5b-4f8f-4395-8fa2-2e7dae5ad256 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:21.282965 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f3d5b-4f8f-4395-8fa2-2e7dae5ad256 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:21.283271 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f3d5b-4f8f-4395-8fa2-2e7dae5ad256 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:21.285187 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f3d5b-4f8f-4395-8fa2-2e7dae5ad256 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:21.296223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 120/242] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:21.302731 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f3d5b-4f8f-4395-8fa2-2e7dae5ad256 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:21.305588 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ae8a0-ca31-4e48-be49-4d18990bebb7 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:21.306106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ae8a0-ca31-4e48-be49-4d18990bebb7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:21.306607 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ae8a0-ca31-4e48-be49-4d18990bebb7 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:21.307999 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 123/243] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:21.314766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9747f-6506-4efa-be65-9f4150ed70d0 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:21.315096 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9747f-6506-4efa-be65-9f4150ed70d0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:21.315312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9747f-6506-4efa-be65-9f4150ed70d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:21.352347 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:21.365188 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-55b9747f-6506-4efa-be65-9f4150ed70d0 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:21.366963 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-55b9747f-6506-4efa-be65-9f4150ed70d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:21.452330 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f3d5b-4f8f-4395-8fa2-2e7dae5ad256 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:21.496322 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f27b5-ca7e-40cb-b837-90957d3be4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:21.497032 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 124/244] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 29 20:19:21.497434 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f27b5-ca7e-40cb-b837-90957d3be4a8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:21.497767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f27b5-ca7e-40cb-b837-90957d3be4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:21.499475 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f27b5-ca7e-40cb-b837-90957d3be4a8 None admin] PATH_INFO: `/v3/users/1fb2d08e87d949798c6b8cccc5e1807d` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:21.522747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f27b5-ca7e-40cb-b837-90957d3be4a8 None admin] RBAC: Authorizing `identity:get_user(user_id=1fb2d08e87d949798c6b8cccc5e1807d)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:21.524512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f27b5-ca7e-40cb-b837-90957d3be4a8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:21.526240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 121/245] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:21 2022] GET /identity/v3/users/1fb2d08e87d949798c6b8cccc5e1807d => generated 297 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:21.532042 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9747f-6506-4efa-be65-9f4150ed70d0 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:21.641090 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9747f-6506-4efa-be65-9f4150ed70d0 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:21.669997 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:21.680788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23000c-2dae-4b30-b080-2a53aea255f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:21.681861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23000c-2dae-4b30-b080-2a53aea255f0 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:21.682736 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23000c-2dae-4b30-b080-2a53aea255f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:21.683336 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23000c-2dae-4b30-b080-2a53aea255f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:21.771405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23000c-2dae-4b30-b080-2a53aea255f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:21.773331 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23000c-2dae-4b30-b080-2a53aea255f0 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:21.882548 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 125/246] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 20:19:21.884008 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f27b5-ca7e-40cb-b837-90957d3be4a8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:21.888375 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23000c-2dae-4b30-b080-2a53aea255f0 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:21.927141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:22.015732 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:22.038533 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f27b5-ca7e-40cb-b837-90957d3be4a8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:22.054933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23000c-2dae-4b30-b080-2a53aea255f0 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:22.083968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e890cb-16d4-4061-addd-3ae495ba8b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:22.084629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e890cb-16d4-4061-addd-3ae495ba8b4d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:22.084905 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e890cb-16d4-4061-addd-3ae495ba8b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:22.085181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e890cb-16d4-4061-addd-3ae495ba8b4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:22.101407 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e890cb-16d4-4061-addd-3ae495ba8b4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:22.107718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-06e890cb-16d4-4061-addd-3ae495ba8b4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:19:22.107718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:22.107718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:22.107718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:22.107718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:22.107718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:22.107718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:22.107718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:22.107718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:22.107718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:22.107718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:22.107718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:22.107718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:22.107718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:22.107718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:22.107718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:22.108662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:22.108662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:22.108662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:22.108662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:22.108662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:22.108662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:22.108662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:22.108662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:22.108662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:22.108662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:22.108662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:22.108662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:22.108662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:22.108662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:22.108662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:22.108662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:22.109488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:22.109488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:22.109488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:22.109488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:22.109488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:22.109488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:22.109488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:22.109488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:22.109488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:22.109488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:22.109488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:19:22.109488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:22.110085 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51149ae-2ce8-440f-aae1-8ce8212814c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:22.110085 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51149ae-2ce8-440f-aae1-8ce8212814c4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:22.110085 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51149ae-2ce8-440f-aae1-8ce8212814c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:22.110337 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51149ae-2ce8-440f-aae1-8ce8212814c4 None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:22.112377 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 122/247] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:19:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:22.112489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51149ae-2ce8-440f-aae1-8ce8212814c4 None admin] RBAC: Authorizing `identity:get_project(project_id=eddee91f1cf44ccf83ca7cd9419ab306)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:22.115019 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51149ae-2ce8-440f-aae1-8ce8212814c4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:22.118691 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 126/248] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:21 2022] GET /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:22.120596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e890cb-16d4-4061-addd-3ae495ba8b4d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:22.124900 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51149ae-2ce8-440f-aae1-8ce8212814c4 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:22.268033 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e890cb-16d4-4061-addd-3ae495ba8b4d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:22.315733 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e59205e-b0d0-4cd7-9237-0c1efaaabab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:22.316372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59205e-b0d0-4cd7-9237-0c1efaaabab2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:22.316653 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59205e-b0d0-4cd7-9237-0c1efaaabab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:22.316925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59205e-b0d0-4cd7-9237-0c1efaaabab2 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:22.317609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e59205e-b0d0-4cd7-9237-0c1efaaabab2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:22.317908 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e59205e-b0d0-4cd7-9237-0c1efaaabab2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:22.319724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e59205e-b0d0-4cd7-9237-0c1efaaabab2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:22.330502 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 123/249] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:22.336280 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e59205e-b0d0-4cd7-9237-0c1efaaabab2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:22.349234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:22.405498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:22.442009 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51149ae-2ce8-440f-aae1-8ce8212814c4 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:22.457214 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e59205e-b0d0-4cd7-9237-0c1efaaabab2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:22.513865 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf57351-e17e-4150-bafd-bc63ee498702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:22.514765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf57351-e17e-4150-bafd-bc63ee498702 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:22.515616 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf57351-e17e-4150-bafd-bc63ee498702 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:22.516369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf57351-e17e-4150-bafd-bc63ee498702 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:22.517772 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf57351-e17e-4150-bafd-bc63ee498702 None admin] RBAC: Adding query filter params (role.id=5ae3132138594bb7a1fbacc9d93198c1, scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306, user.id=1fb2d08e87d949798c6b8cccc5e1807d) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:22.518573 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c5f58-58a7-4a4d-a22e-dde77f3c560b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:22.519058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf57351-e17e-4150-bafd-bc63ee498702 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:22.519834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c5f58-58a7-4a4d-a22e-dde77f3c560b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:22.521559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf57351-e17e-4150-bafd-bc63ee498702 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:22.521994 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c5f58-58a7-4a4d-a22e-dde77f3c560b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:22.522427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c5f58-58a7-4a4d-a22e-dde77f3c560b None admin] PATH_INFO: `/v3/users/5f2be18779f74c668807d3743358c46c` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:22.525418 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c5f58-58a7-4a4d-a22e-dde77f3c560b None admin] RBAC: Authorizing `identity:get_user(user_id=5f2be18779f74c668807d3743358c46c)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:22.527101 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c5f58-58a7-4a4d-a22e-dde77f3c560b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:22.528824 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 124/250] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:22 2022] GET /identity/v3/users/5f2be18779f74c668807d3743358c46c => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:22.530010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58820d6-6928-40a8-853c-0e7342439474 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:22.530285 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58820d6-6928-40a8-853c-0e7342439474 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:22.530685 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58820d6-6928-40a8-853c-0e7342439474 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:22.532118 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 125/251] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:22.533310 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba5570-b589-4d04-932c-e19fb18abde1 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:22.533965 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba5570-b589-4d04-932c-e19fb18abde1 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:22.534444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba5570-b589-4d04-932c-e19fb18abde1 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:22.535971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 126/252] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:22.543959 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba5570-b589-4d04-932c-e19fb18abde1 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:22.544164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 127/253] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:19:22 2022] GET /identity/v3/role_assignments?user.id=1fb2d08e87d949798c6b8cccc5e1807d&scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306&role.id=5ae3132138594bb7a1fbacc9d93198c1 => generated 259 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:22.544164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2f837-d697-4400-96a5-5dfef18da68f None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:22.544164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2f837-d697-4400-96a5-5dfef18da68f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:22.544164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2f837-d697-4400-96a5-5dfef18da68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:22.572050 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-bbb2f837-d697-4400-96a5-5dfef18da68f None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:22.572050 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-bbb2f837-d697-4400-96a5-5dfef18da68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:22.720415 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 128/254] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:22.721271 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0efef-4809-496b-9fce-cf41c4f5f425 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:22.721667 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0efef-4809-496b-9fce-cf41c4f5f425 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:22.722058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd0efef-4809-496b-9fce-cf41c4f5f425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:22.723968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:22.748026 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-cdd0efef-4809-496b-9fce-cf41c4f5f425 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:22.748026 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-cdd0efef-4809-496b-9fce-cf41c4f5f425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:22.788987 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba5570-b589-4d04-932c-e19fb18abde1 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:22.868405 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe725d8-6fb0-4718-bf82-05ce48f1e1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:22.869361 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe725d8-6fb0-4718-bf82-05ce48f1e1b6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:22.869830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe725d8-6fb0-4718-bf82-05ce48f1e1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:22.870310 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe725d8-6fb0-4718-bf82-05ce48f1e1b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:22.943586 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe725d8-6fb0-4718-bf82-05ce48f1e1b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:22.945447 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe725d8-6fb0-4718-bf82-05ce48f1e1b6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:22.949862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 129/255] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:23.044521 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:23.092439 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 127/256] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:23.098115 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0efef-4809-496b-9fce-cf41c4f5f425 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:23.123224 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:23.191560 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe725d8-6fb0-4718-bf82-05ce48f1e1b6 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:23.235098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd0efef-4809-496b-9fce-cf41c4f5f425 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:23.280200 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91035e3f-ff5e-4372-8854-2159d9454e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:23.281221 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91035e3f-ff5e-4372-8854-2159d9454e26 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:23.281719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91035e3f-ff5e-4372-8854-2159d9454e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:23.282178 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91035e3f-ff5e-4372-8854-2159d9454e26 None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:23.288889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91035e3f-ff5e-4372-8854-2159d9454e26 None admin] RBAC: Authorizing `identity:get_project(project_id=eddee91f1cf44ccf83ca7cd9419ab306)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:23.291190 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91035e3f-ff5e-4372-8854-2159d9454e26 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:23.293479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 130/257] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:23 2022] GET /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:23.309217 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91035e3f-ff5e-4372-8854-2159d9454e26 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:23.414816 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe725d8-6fb0-4718-bf82-05ce48f1e1b6 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:23.445603 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:23.462425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91035e3f-ff5e-4372-8854-2159d9454e26 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:23.501170 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba935f-6437-40aa-9fc9-75f5ffaf7060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:23.501820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba935f-6437-40aa-9fc9-75f5ffaf7060 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:23.502097 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba935f-6437-40aa-9fc9-75f5ffaf7060 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:23.502369 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba935f-6437-40aa-9fc9-75f5ffaf7060 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:23.521314 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdba935f-6437-40aa-9fc9-75f5ffaf7060 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:23.528679 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b31fa2-007f-4c74-859b-4067b74514ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:23.529494 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b31fa2-007f-4c74-859b-4067b74514ed None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:23.529996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b31fa2-007f-4c74-859b-4067b74514ed None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:23.530465 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b31fa2-007f-4c74-859b-4067b74514ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:23.531460 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b31fa2-007f-4c74-859b-4067b74514ed None admin] RBAC: Adding query filter params (role.id=322da7bd64024770a64bdb73e5c54719, scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306, user.id=5f2be18779f74c668807d3743358c46c) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:23.532017 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b31fa2-007f-4c74-859b-4067b74514ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:23.532348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-bdba935f-6437-40aa-9fc9-75f5ffaf7060 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:23.532348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:23.532348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:23.532348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:23.532348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:23.532348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:23.532348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:23.532348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:23.532348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:23.532348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:23.532348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:23.532348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:23.532348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:23.532348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:23.532348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:23.532348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:23.533191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:23.533191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:23.533191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:23.533191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:23.533191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:23.533191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:23.533191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:23.533191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:23.533191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:23.533191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:23.533191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:23.533191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:23.533191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:23.533191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:23.533191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:23.533191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:23.534232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:23.534232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:23.534232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:23.534232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:23.534232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:23.534232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:23.534232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:23.534232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:23.534232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:23.534232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:23.534232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:23.534232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:23.534831 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 128/258] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:19:23.535525 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba935f-6437-40aa-9fc9-75f5ffaf7060 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:23.536997 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b31fa2-007f-4c74-859b-4067b74514ed None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:23.562183 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 131/259] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:19:23 2022] GET /identity/v3/role_assignments?user.id=5f2be18779f74c668807d3743358c46c&scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306&role.id=322da7bd64024770a64bdb73e5c54719 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:23.563049 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b31fa2-007f-4c74-859b-4067b74514ed None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:23.588124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:23.702690 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba935f-6437-40aa-9fc9-75f5ffaf7060 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:23.741128 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c25ecf-9979-41fb-9c4d-cf9c3ce36c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:23.741799 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c25ecf-9979-41fb-9c4d-cf9c3ce36c29 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:23.742080 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c25ecf-9979-41fb-9c4d-cf9c3ce36c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:23.742355 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c25ecf-9979-41fb-9c4d-cf9c3ce36c29 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:23.753983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c25ecf-9979-41fb-9c4d-cf9c3ce36c29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:23.759620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-a8c25ecf-9979-41fb-9c4d-cf9c3ce36c29 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:23.759620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:23.759620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:23.759620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:23.759620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:23.759620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:23.759620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:23.759620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:23.759620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:23.759620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:23.759620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:23.759620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:23.759620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:23.759620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:23.759620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:23.759620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:23.760772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:23.760772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:23.760772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:23.760772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:23.760772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:23.760772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:23.760772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:23.760772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:23.760772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:23.760772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:23.760772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:23.760772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:23.760772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:23.760772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:23.760772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:23.760772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:23.761632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:23.761632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:23.761632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:23.761632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:23.761632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:23.761632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:23.761632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:23.761632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:23.761632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:23.761632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:23.761632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:23.761632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:23.763600 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 129/260] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:19:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:23.769773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c25ecf-9979-41fb-9c4d-cf9c3ce36c29 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:23.790227 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b31fa2-007f-4c74-859b-4067b74514ed None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:23.875882 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bc601-4a64-4b40-8786-225e74cf1c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:23.876823 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bc601-4a64-4b40-8786-225e74cf1c1c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:23.877423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bc601-4a64-4b40-8786-225e74cf1c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:23.877838 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bc601-4a64-4b40-8786-225e74cf1c1c None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:23.878704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8bc601-4a64-4b40-8786-225e74cf1c1c None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:23.879233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8bc601-4a64-4b40-8786-225e74cf1c1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:23.880807 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8bc601-4a64-4b40-8786-225e74cf1c1c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:23.900083 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 132/261] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:23.906013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bc601-4a64-4b40-8786-225e74cf1c1c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:23.924315 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:23.929527 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:23.929856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c25ecf-9979-41fb-9c4d-cf9c3ce36c29 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:23.972241 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c772f4-0aff-47de-a46c-3ad909857910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:23.972788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c772f4-0aff-47de-a46c-3ad909857910 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:23.973069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c772f4-0aff-47de-a46c-3ad909857910 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:23.975959 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c772f4-0aff-47de-a46c-3ad909857910 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:23.975959 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c772f4-0aff-47de-a46c-3ad909857910 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:23.975959 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c772f4-0aff-47de-a46c-3ad909857910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:23.975959 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c772f4-0aff-47de-a46c-3ad909857910 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:23.992777 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 130/262] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:19:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:23.998500 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c772f4-0aff-47de-a46c-3ad909857910 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:24.115734 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c772f4-0aff-47de-a46c-3ad909857910 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:24.116360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bc601-4a64-4b40-8786-225e74cf1c1c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:24.154349 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1d634-0fda-4b67-b1a4-d11e6c39c265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:24.154992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1d634-0fda-4b67-b1a4-d11e6c39c265 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:24.155278 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1d634-0fda-4b67-b1a4-d11e6c39c265 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:24.155556 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1d634-0fda-4b67-b1a4-d11e6c39c265 None admin] PATH_INFO: `/v3/users/5f2be18779f74c668807d3743358c46c` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:24.157219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1d634-0fda-4b67-b1a4-d11e6c39c265 None admin] RBAC: Authorizing `identity:get_user(user_id=5f2be18779f74c668807d3743358c46c)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:24.158848 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1d634-0fda-4b67-b1a4-d11e6c39c265 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:24.161281 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 131/263] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:23 2022] GET /identity/v3/users/5f2be18779f74c668807d3743358c46c => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:24.167166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1d634-0fda-4b67-b1a4-d11e6c39c265 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:24.186244 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdafdb-91df-4c06-ba80-70bcfa22db99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:24.187143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdafdb-91df-4c06-ba80-70bcfa22db99 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:24.187592 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdafdb-91df-4c06-ba80-70bcfa22db99 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:24.188043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdafdb-91df-4c06-ba80-70bcfa22db99 None admin] PATH_INFO: `/v3/users/01e34eec23f243ebafc5a3257a700356` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:24.189369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fdafdb-91df-4c06-ba80-70bcfa22db99 None admin] RBAC: Authorizing `identity:get_user(user_id=01e34eec23f243ebafc5a3257a700356)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:24.190996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fdafdb-91df-4c06-ba80-70bcfa22db99 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:24.193404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 133/264] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:23 2022] GET /identity/v3/users/01e34eec23f243ebafc5a3257a700356 => generated 297 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:24.198962 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdafdb-91df-4c06-ba80-70bcfa22db99 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:24.235379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:24.334715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:24.362534 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdafdb-91df-4c06-ba80-70bcfa22db99 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:24.383410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1d634-0fda-4b67-b1a4-d11e6c39c265 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:24.419222 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0782a1-ef0a-49c8-8a2b-74c9cbe187f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:24.420197 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0782a1-ef0a-49c8-8a2b-74c9cbe187f7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:24.420497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0782a1-ef0a-49c8-8a2b-74c9cbe187f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:24.420789 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0782a1-ef0a-49c8-8a2b-74c9cbe187f7 None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:24.422646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0782a1-ef0a-49c8-8a2b-74c9cbe187f7 None admin] RBAC: Authorizing `identity:get_project(project_id=eddee91f1cf44ccf83ca7cd9419ab306)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:24.424136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0782a1-ef0a-49c8-8a2b-74c9cbe187f7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:24.426084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 134/265] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:24 2022] GET /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:24.426800 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb26e0c-f889-4587-a125-c0cdf13d2f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:24.428129 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb26e0c-f889-4587-a125-c0cdf13d2f40 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:24.428611 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb26e0c-f889-4587-a125-c0cdf13d2f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:24.429109 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb26e0c-f889-4587-a125-c0cdf13d2f40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:24.432657 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2f59d-9616-428c-8367-12481e2545e0 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:24.432952 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2f59d-9616-428c-8367-12481e2545e0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:24.433238 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f2f59d-9616-428c-8367-12481e2545e0 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:24.435564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 135/266] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:24.440696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2f59d-9616-428c-8367-12481e2545e0 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:24.522324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb26e0c-f889-4587-a125-c0cdf13d2f40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:24.524654 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb26e0c-f889-4587-a125-c0cdf13d2f40 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:24.574996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:24.584573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f2f59d-9616-428c-8367-12481e2545e0 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:24.634849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a0a50-0063-42db-b7ba-b3a10c42ff6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:24.636084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a0a50-0063-42db-b7ba-b3a10c42ff6a None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:24.636508 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a0a50-0063-42db-b7ba-b3a10c42ff6a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:24.637153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a0a50-0063-42db-b7ba-b3a10c42ff6a None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306/users/01e34eec23f243ebafc5a3257a700356/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:24.640073 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97a0a50-0063-42db-b7ba-b3a10c42ff6a None admin] RBAC: Authorizing `identity:create_grant(project_id=eddee91f1cf44ccf83ca7cd9419ab306, user_id=01e34eec23f243ebafc5a3257a700356, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:24.641686 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97a0a50-0063-42db-b7ba-b3a10c42ff6a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:24.647307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 132/267] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:24.648986 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853bd38-85c9-42c1-bc2b-5e8014ae284e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:24.649454 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853bd38-85c9-42c1-bc2b-5e8014ae284e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:24.649845 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853bd38-85c9-42c1-bc2b-5e8014ae284e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:24.662042 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 136/268] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:19:24 2022] PUT /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306/users/01e34eec23f243ebafc5a3257a700356/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:19:24.663262 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a0a50-0063-42db-b7ba-b3a10c42ff6a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:24.671260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-9853bd38-85c9-42c1-bc2b-5e8014ae284e None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:24.672651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9853bd38-85c9-42c1-bc2b-5e8014ae284e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:24.675441 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:24.760630 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 133/269] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:24.768257 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e594c-7dcc-4445-8de7-580d9ab00a19 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:24.768722 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e594c-7dcc-4445-8de7-580d9ab00a19 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:24.769128 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e594c-7dcc-4445-8de7-580d9ab00a19 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:24.771152 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 134/270] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:24.779052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9548e1-a459-4800-aca2-913c0456db01 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:24.779555 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9548e1-a459-4800-aca2-913c0456db01 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:24.780013 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9548e1-a459-4800-aca2-913c0456db01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:24.786135 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97a0a50-0063-42db-b7ba-b3a10c42ff6a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:24.820339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5c9548e1-a459-4800-aca2-913c0456db01 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:24.822119 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5c9548e1-a459-4800-aca2-913c0456db01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:24.840529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f327b8-135b-43b8-b155-082cb2fdf328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:24.841192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f327b8-135b-43b8-b155-082cb2fdf328 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:24.841473 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f327b8-135b-43b8-b155-082cb2fdf328 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:24.841748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f327b8-135b-43b8-b155-082cb2fdf328 None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:24.844425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f327b8-135b-43b8-b155-082cb2fdf328 None admin] RBAC: Authorizing `identity:get_project(project_id=eddee91f1cf44ccf83ca7cd9419ab306)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:24.845912 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f327b8-135b-43b8-b155-082cb2fdf328 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:24.850502 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 137/271] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:24 2022] GET /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:24.865668 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f327b8-135b-43b8-b155-082cb2fdf328 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:24.926313 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:24.952838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 135/272] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:19:24.968700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:25.005114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f327b8-135b-43b8-b155-082cb2fdf328 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:25.051473 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89cf5c1-a299-445e-93e1-bebba9ea552e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:25.052124 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cf5c1-a299-445e-93e1-bebba9ea552e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:25.052413 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cf5c1-a299-445e-93e1-bebba9ea552e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:25.052694 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cf5c1-a299-445e-93e1-bebba9ea552e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:25.053437 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89cf5c1-a299-445e-93e1-bebba9ea552e None admin] RBAC: Adding query filter params (role.id=578c9a17c0fa434f9602f5d2b3648785, scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306, user.id=5f2be18779f74c668807d3743358c46c) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:25.053754 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89cf5c1-a299-445e-93e1-bebba9ea552e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:25.055208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89cf5c1-a299-445e-93e1-bebba9ea552e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:25.062638 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 138/273] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:19:24 2022] GET /identity/v3/role_assignments?user.id=5f2be18779f74c668807d3743358c46c&scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306&role.id=578c9a17c0fa434f9602f5d2b3648785 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:25.109287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9548e1-a459-4800-aca2-913c0456db01 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:25.229735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f1789-4947-4032-81a5-ef08156e8b4e None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:25.230046 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f1789-4947-4032-81a5-ef08156e8b4e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:25.230305 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f1789-4947-4032-81a5-ef08156e8b4e None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:25.232000 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 139/274] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:25.241427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad71f45-2923-4585-9e84-268e93e18ddb None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:25.241767 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad71f45-2923-4585-9e84-268e93e18ddb None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:25.241986 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad71f45-2923-4585-9e84-268e93e18ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:25.264382 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-cad71f45-2923-4585-9e84-268e93e18ddb None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:25.265965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-cad71f45-2923-4585-9e84-268e93e18ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:25.273735 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9548e1-a459-4800-aca2-913c0456db01 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:25.319506 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386cd2c-48ab-4c5d-9bfb-4f2be2bf404f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:25.320195 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386cd2c-48ab-4c5d-9bfb-4f2be2bf404f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:25.320476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386cd2c-48ab-4c5d-9bfb-4f2be2bf404f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:25.320748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386cd2c-48ab-4c5d-9bfb-4f2be2bf404f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:25.335045 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386cd2c-48ab-4c5d-9bfb-4f2be2bf404f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:25.341030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-6386cd2c-48ab-4c5d-9bfb-4f2be2bf404f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:25.341030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:25.341030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:25.341030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:25.341030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:25.341030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:25.341030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:25.341030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:25.341030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:25.341030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:25.341030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:25.341030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:25.341030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:25.341030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:25.341030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:25.341030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:25.341949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:25.341949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:25.341949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:25.341949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:25.341949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:25.341949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:25.341949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:25.341949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:25.341949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:25.341949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:25.341949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:25.341949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:25.341949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:25.341949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:25.341949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:25.341949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:25.342776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:25.342776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:25.342776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:25.342776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:25.342776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:25.342776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:25.342776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:25.342776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:25.342776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:25.342776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:25.342776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:25.342776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:25.342776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 136/275] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:25.351128 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386cd2c-48ab-4c5d-9bfb-4f2be2bf404f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:25.368866 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 140/276] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:19:25.369755 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad71f45-2923-4585-9e84-268e93e18ddb None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:25.416326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:25.451744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:25.466259 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386cd2c-48ab-4c5d-9bfb-4f2be2bf404f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:25.492372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad71f45-2923-4585-9e84-268e93e18ddb None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:25.516825 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06d857-f16c-4796-90ea-587352f7fba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:25.517676 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06d857-f16c-4796-90ea-587352f7fba0 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:25.518135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06d857-f16c-4796-90ea-587352f7fba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:25.518545 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06d857-f16c-4796-90ea-587352f7fba0 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:25.519550 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06d857-f16c-4796-90ea-587352f7fba0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:25.520222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06d857-f16c-4796-90ea-587352f7fba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:25.522573 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06d857-f16c-4796-90ea-587352f7fba0 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:25.534518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 137/277] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:25.542307 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd16c4-95cd-4ce2-aaa6-06c7af519ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:25.543929 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06d857-f16c-4796-90ea-587352f7fba0 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:25.545474 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd16c4-95cd-4ce2-aaa6-06c7af519ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:25.545891 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd16c4-95cd-4ce2-aaa6-06c7af519ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:25.546295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd16c4-95cd-4ce2-aaa6-06c7af519ab8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:25.548111 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd16c4-95cd-4ce2-aaa6-06c7af519ab8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:25.550436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd16c4-95cd-4ce2-aaa6-06c7af519ab8 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:25.552703 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 141/278] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:19:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:25.565066 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd16c4-95cd-4ce2-aaa6-06c7af519ab8 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:25.672739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06d857-f16c-4796-90ea-587352f7fba0 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:25.684799 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:25.695790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd16c4-95cd-4ce2-aaa6-06c7af519ab8 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:25.718860 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba963a4-d485-418a-8690-a6f01159960e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:25.719586 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba963a4-d485-418a-8690-a6f01159960e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:25.719866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba963a4-d485-418a-8690-a6f01159960e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:25.720229 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba963a4-d485-418a-8690-a6f01159960e None admin] PATH_INFO: `/v3/users/2abdbfef38d147b3805cd5be2e10783d` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:25.732262 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0cddc-db1f-463a-9051-5c5bad9d107d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:25.732883 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0cddc-db1f-463a-9051-5c5bad9d107d None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:25.733171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0cddc-db1f-463a-9051-5c5bad9d107d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:25.733475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0cddc-db1f-463a-9051-5c5bad9d107d None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:25.734133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0cddc-db1f-463a-9051-5c5bad9d107d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:25.735329 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba963a4-d485-418a-8690-a6f01159960e None admin] RBAC: Authorizing `identity:get_user(user_id=2abdbfef38d147b3805cd5be2e10783d)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:25.735573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0cddc-db1f-463a-9051-5c5bad9d107d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:25.737106 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba963a4-d485-418a-8690-a6f01159960e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:25.739211 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 138/279] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:25 2022] GET /identity/v3/users/2abdbfef38d147b3805cd5be2e10783d => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:25.740160 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba963a4-d485-418a-8690-a6f01159960e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:25.745419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:25.759716 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 142/280] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 29 20:19:25 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:19:25.760593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0cddc-db1f-463a-9051-5c5bad9d107d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:25.890580 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba963a4-d485-418a-8690-a6f01159960e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:25.900133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0cddc-db1f-463a-9051-5c5bad9d107d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:25.940733 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b35ba9-b203-4e92-99d8-c0b750436647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:25.941498 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b35ba9-b203-4e92-99d8-c0b750436647 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:25.942011 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b35ba9-b203-4e92-99d8-c0b750436647 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:25.942286 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b35ba9-b203-4e92-99d8-c0b750436647 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:25.942592 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a912a-de41-40b9-bc57-c7dd68c343ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:25.943507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a912a-de41-40b9-bc57-c7dd68c343ff None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:25.943969 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a912a-de41-40b9-bc57-c7dd68c343ff None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:25.944730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a912a-de41-40b9-bc57-c7dd68c343ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:25.946134 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a912a-de41-40b9-bc57-c7dd68c343ff None admin] RBAC: Adding query filter params (role.id=89cf5ed93cc642588a54639342c59ea6, user.id=2abdbfef38d147b3805cd5be2e10783d, scope.system=all) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:25.947785 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a912a-de41-40b9-bc57-c7dd68c343ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:25.951042 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a912a-de41-40b9-bc57-c7dd68c343ff None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:25.955823 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b35ba9-b203-4e92-99d8-c0b750436647 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:25.959230 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 143/281] 10.222.0.18 () {56 vars in 1308 bytes} [Mon Aug 29 20:19:25 2022] GET /identity/v3/role_assignments?user.id=2abdbfef38d147b3805cd5be2e10783d&scope.system=all&role.id=89cf5ed93cc642588a54639342c59ea6 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:25.961166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-c0b35ba9-b203-4e92-99d8-c0b750436647 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:19:25.961166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:25.961166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:25.961166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:25.961166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:25.961166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:25.961166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:25.961166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:25.961166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:25.961166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:25.961166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:25.961166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:25.961166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:25.961166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:25.961166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:25.961166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:25.962169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:25.962169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:25.962169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:25.962169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:25.962169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:25.962169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:25.962169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:25.962169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:25.962169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:25.962169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:25.962169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:25.962169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:25.962169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:25.962169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:25.962169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:25.962169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:25.962956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:25.962956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:25.962956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:25.962956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:25.962956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:25.962956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:25.962956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:25.962956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:25.962956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:25.962956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:25.962956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:19:25.962956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:25.962956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 139/282] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:25.972114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a912a-de41-40b9-bc57-c7dd68c343ff None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:26.042491 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:26.099658 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a912a-de41-40b9-bc57-c7dd68c343ff None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:26.147653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545ef67-6b86-452d-a571-a5b9cb61ba7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:26.148295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545ef67-6b86-452d-a571-a5b9cb61ba7b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:26.148577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545ef67-6b86-452d-a571-a5b9cb61ba7b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:26.148850 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545ef67-6b86-452d-a571-a5b9cb61ba7b None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:26.149528 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545ef67-6b86-452d-a571-a5b9cb61ba7b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:26.150443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545ef67-6b86-452d-a571-a5b9cb61ba7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:26.152322 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545ef67-6b86-452d-a571-a5b9cb61ba7b None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:26.164011 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 144/283] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:26.170542 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b35ba9-b203-4e92-99d8-c0b750436647 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:26.214496 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:26.326051 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b35ba9-b203-4e92-99d8-c0b750436647 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:26.371726 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabdd2a-b8f2-4023-9aa2-6fa6695af8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:26.372404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabdd2a-b8f2-4023-9aa2-6fa6695af8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:26.372748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabdd2a-b8f2-4023-9aa2-6fa6695af8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:26.373029 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabdd2a-b8f2-4023-9aa2-6fa6695af8d1 None admin] PATH_INFO: `/v3/users/1fb2d08e87d949798c6b8cccc5e1807d` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:26.374190 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabdd2a-b8f2-4023-9aa2-6fa6695af8d1 None admin] RBAC: Authorizing `identity:get_user(user_id=1fb2d08e87d949798c6b8cccc5e1807d)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:26.375651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabdd2a-b8f2-4023-9aa2-6fa6695af8d1 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:26.377610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 140/284] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:26 2022] GET /identity/v3/users/1fb2d08e87d949798c6b8cccc5e1807d => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:26.383721 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545ef67-6b86-452d-a571-a5b9cb61ba7b None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:26.522940 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545ef67-6b86-452d-a571-a5b9cb61ba7b None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:26.549894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:26.575384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f01c4-a93d-4242-afd4-7691de6400b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:26.576078 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f01c4-a93d-4242-afd4-7691de6400b6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:26.576366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f01c4-a93d-4242-afd4-7691de6400b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:26.576712 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f01c4-a93d-4242-afd4-7691de6400b6 None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:26.577907 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f01c4-a93d-4242-afd4-7691de6400b6 None admin] RBAC: Authorizing `identity:get_project(project_id=eddee91f1cf44ccf83ca7cd9419ab306)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:26.579417 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f01c4-a93d-4242-afd4-7691de6400b6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:26.581422 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 145/285] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:26 2022] GET /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:26.587251 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabdd2a-b8f2-4023-9aa2-6fa6695af8d1 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:26.744666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:26.751017 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabdd2a-b8f2-4023-9aa2-6fa6695af8d1 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:26.804829 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdec327-7267-40a5-880d-b49b97f50c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:26.805416 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdec327-7267-40a5-880d-b49b97f50c07 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:26.805696 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdec327-7267-40a5-880d-b49b97f50c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:26.806004 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdec327-7267-40a5-880d-b49b97f50c07 None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306/users/1fb2d08e87d949798c6b8cccc5e1807d/roles/5ae3132138594bb7a1fbacc9d93198c1` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:26.813289 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdec327-7267-40a5-880d-b49b97f50c07 None admin] RBAC: Authorizing `identity:create_grant(project_id=eddee91f1cf44ccf83ca7cd9419ab306, user_id=1fb2d08e87d949798c6b8cccc5e1807d, role_id=5ae3132138594bb7a1fbacc9d93198c1)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:26.814761 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdec327-7267-40a5-880d-b49b97f50c07 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:26.840472 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 141/286] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:19:26 2022] PUT /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306/users/1fb2d08e87d949798c6b8cccc5e1807d/roles/5ae3132138594bb7a1fbacc9d93198c1 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:19:27.129425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199209e-6585-4aa0-9ae5-3df82f307417 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:27.129934 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199209e-6585-4aa0-9ae5-3df82f307417 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:27.129934 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199209e-6585-4aa0-9ae5-3df82f307417 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:27.131424 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 146/287] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:27.140696 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c1b59-f3a4-4c67-87b3-475e42a968c7 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:27.141002 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c1b59-f3a4-4c67-87b3-475e42a968c7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:27.141279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c1b59-f3a4-4c67-87b3-475e42a968c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:27.166981 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d92c1b59-f3a4-4c67-87b3-475e42a968c7 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:27.168267 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d92c1b59-f3a4-4c67-87b3-475e42a968c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:27.246692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 142/288] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:27.682431 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199209e-6585-4aa0-9ae5-3df82f307417 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:27.825195 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199209e-6585-4aa0-9ae5-3df82f307417 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:27.874192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc6133-c83c-4e74-b66a-52fbcf60ab2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:27.875142 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc6133-c83c-4e74-b66a-52fbcf60ab2c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:27.875568 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc6133-c83c-4e74-b66a-52fbcf60ab2c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:27.875998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc6133-c83c-4e74-b66a-52fbcf60ab2c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:27.901457 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc6133-c83c-4e74-b66a-52fbcf60ab2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:27.911964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-f1cc6133-c83c-4e74-b66a-52fbcf60ab2c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:27.911964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:27.911964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:27.911964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:27.911964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:27.911964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:27.911964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:27.911964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:27.911964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:27.911964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:27.911964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:27.911964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:27.911964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:27.911964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:27.911964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:27.911964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:27.912933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:27.912933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:27.912933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:27.912933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:27.912933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:27.912933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:27.912933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:27.912933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:27.912933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:27.912933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:27.912933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:27.912933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:27.912933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:27.912933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:27.912933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:27.912933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:27.913783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:27.913783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:27.913783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:27.913783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:27.913783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:27.913783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:27.913783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:27.913783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:27.913783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:27.913783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:27.913783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:27.913783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:27.914832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 147/289] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:27.921220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c1b59-f3a4-4c67-87b3-475e42a968c7 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:28.027856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:28.050766 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c1b59-f3a4-4c67-87b3-475e42a968c7 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:28.101286 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067b233-a3ee-4a00-861b-6e73327eef92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:28.101918 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067b233-a3ee-4a00-861b-6e73327eef92 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:28.102205 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067b233-a3ee-4a00-861b-6e73327eef92 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:28.102482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067b233-a3ee-4a00-861b-6e73327eef92 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:28.103188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067b233-a3ee-4a00-861b-6e73327eef92 None admin] RBAC: Adding query filter params (name=member) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:28.103542 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067b233-a3ee-4a00-861b-6e73327eef92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:28.105139 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067b233-a3ee-4a00-861b-6e73327eef92 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:28.120359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 143/290] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:28.127143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc6133-c83c-4e74-b66a-52fbcf60ab2c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:28.191670 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:28.266618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc6133-c83c-4e74-b66a-52fbcf60ab2c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:28.326740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca78be-293c-409e-974f-dd75edc9d9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:28.327436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca78be-293c-409e-974f-dd75edc9d9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:28.327847 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca78be-293c-409e-974f-dd75edc9d9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:28.328156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca78be-293c-409e-974f-dd75edc9d9d1 None admin] PATH_INFO: `/v3/users/01e34eec23f243ebafc5a3257a700356` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:28.329371 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca78be-293c-409e-974f-dd75edc9d9d1 None admin] RBAC: Authorizing `identity:get_user(user_id=01e34eec23f243ebafc5a3257a700356)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:28.330996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca78be-293c-409e-974f-dd75edc9d9d1 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:28.333080 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 148/291] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:28 2022] GET /identity/v3/users/01e34eec23f243ebafc5a3257a700356 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:28.339090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067b233-a3ee-4a00-861b-6e73327eef92 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:28.395031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0d130-2b46-4af6-a208-70abfcfbb04b None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:28.395352 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0d130-2b46-4af6-a208-70abfcfbb04b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:28.395594 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0d130-2b46-4af6-a208-70abfcfbb04b None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:28.397269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 149/292] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:19:28.410433 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5770d-3a41-412c-a29f-23dc54fd6bbc None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:28.410706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5770d-3a41-412c-a29f-23dc54fd6bbc None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:28.410989 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5770d-3a41-412c-a29f-23dc54fd6bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:28.445145 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-97c5770d-3a41-412c-a29f-23dc54fd6bbc None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:28.446363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-97c5770d-3a41-412c-a29f-23dc54fd6bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:28.497143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067b233-a3ee-4a00-861b-6e73327eef92 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:28.569475 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd9f9c-bcf2-40a2-b613-f08cdf1ebf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:28.571317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd9f9c-bcf2-40a2-b613-f08cdf1ebf54 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:28.571623 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd9f9c-bcf2-40a2-b613-f08cdf1ebf54 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:28.572649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd9f9c-bcf2-40a2-b613-f08cdf1ebf54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:28.579252 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 150/293] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:28.594381 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:28.605822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33fe38-c799-47b5-8602-17b3ae6ef440 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:28.606365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33fe38-c799-47b5-8602-17b3ae6ef440 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:28.606822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33fe38-c799-47b5-8602-17b3ae6ef440 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:28.608731 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 151/294] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:28.619735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db525bc4-b821-4792-b611-6e78de4ee4f7 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:28.620285 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db525bc4-b821-4792-b611-6e78de4ee4f7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:28.620727 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db525bc4-b821-4792-b611-6e78de4ee4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:28.661085 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-db525bc4-b821-4792-b611-6e78de4ee4f7 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:28.662757 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-db525bc4-b821-4792-b611-6e78de4ee4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:28.682588 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd9f9c-bcf2-40a2-b613-f08cdf1ebf54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:28.684508 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd9f9c-bcf2-40a2-b613-f08cdf1ebf54 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:28.750051 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:28.787958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 152/295] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:28.787958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8befd57-75a8-4a9b-8018-abe473d11cb2 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:28.787958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8befd57-75a8-4a9b-8018-abe473d11cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:28.787958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8befd57-75a8-4a9b-8018-abe473d11cb2 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:28.787958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 153/296] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:28.796097 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba4a12-a3fc-45d6-9957-f991fa1c2768 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:28.796386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba4a12-a3fc-45d6-9957-f991fa1c2768 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:28.796661 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba4a12-a3fc-45d6-9957-f991fa1c2768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:28.830119 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-56ba4a12-a3fc-45d6-9957-f991fa1c2768 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:28.831236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-56ba4a12-a3fc-45d6-9957-f991fa1c2768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:28.861338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 144/297] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:28.869825 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd9f9c-bcf2-40a2-b613-f08cdf1ebf54 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:28.948817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 154/298] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:28.949699 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba4a12-a3fc-45d6-9957-f991fa1c2768 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:28.950478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:29.019046 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd9f9c-bcf2-40a2-b613-f08cdf1ebf54 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:29.063285 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c7f3d-b17d-4296-98cb-b775f9e0e1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:29.064312 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c7f3d-b17d-4296-98cb-b775f9e0e1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:29.064810 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c7f3d-b17d-4296-98cb-b775f9e0e1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:29.065305 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c7f3d-b17d-4296-98cb-b775f9e0e1a7 None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:29.067109 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c7f3d-b17d-4296-98cb-b775f9e0e1a7 None admin] RBAC: Authorizing `identity:get_project(project_id=eddee91f1cf44ccf83ca7cd9419ab306)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:29.068935 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c7f3d-b17d-4296-98cb-b775f9e0e1a7 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:29.071336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 145/299] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:28 2022] GET /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:29.078462 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c7f3d-b17d-4296-98cb-b775f9e0e1a7 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:29.116670 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba4a12-a3fc-45d6-9957-f991fa1c2768 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:29.121770 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:29.169308 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c69f7-0478-4827-a0bc-ce192fafe933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:29.170232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c69f7-0478-4827-a0bc-ce192fafe933 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:29.170517 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c69f7-0478-4827-a0bc-ce192fafe933 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:29.170794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c69f7-0478-4827-a0bc-ce192fafe933 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:29.182149 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c69f7-0478-4827-a0bc-ce192fafe933 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:29.187143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-460c69f7-0478-4827-a0bc-ce192fafe933 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:29.187143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:29.187143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:29.187143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:29.187143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:29.187143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:29.187143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:29.187143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:29.187143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:29.187143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:29.187143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:29.187143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:29.187143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:29.187143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:29.187143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:29.187143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:29.188057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:29.188057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:29.188057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:29.188057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:29.188057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:29.188057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:29.188057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:29.188057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:29.188057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:29.188057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:29.188057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:29.188057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:29.188057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:29.188057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:29.188057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:29.188057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:29.188857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:29.188857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:29.188857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:29.188857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:29.188857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:29.188857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:29.188857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:29.188857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:29.188857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:29.188857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:29.188857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:29.188857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:29.188857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 155/300] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:29.189493 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c69f7-0478-4827-a0bc-ce192fafe933 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:29.210470 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c7f3d-b17d-4296-98cb-b775f9e0e1a7 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:29.228962 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:29.253598 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf9c6f-9dbb-4ae2-9441-3ec3aa596bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:29.254319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf9c6f-9dbb-4ae2-9441-3ec3aa596bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:29.254570 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf9c6f-9dbb-4ae2-9441-3ec3aa596bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:29.254950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf9c6f-9dbb-4ae2-9441-3ec3aa596bf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:29.255478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf9c6f-9dbb-4ae2-9441-3ec3aa596bf4 None admin] RBAC: Adding query filter params (role.id=89cf5ed93cc642588a54639342c59ea6, scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306, user.id=01e34eec23f243ebafc5a3257a700356) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:29.255856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf9c6f-9dbb-4ae2-9441-3ec3aa596bf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:29.257255 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf9c6f-9dbb-4ae2-9441-3ec3aa596bf4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:29.265215 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 146/301] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:19:29 2022] GET /identity/v3/role_assignments?user.id=01e34eec23f243ebafc5a3257a700356&scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306&role.id=89cf5ed93cc642588a54639342c59ea6 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:29.277032 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf9c6f-9dbb-4ae2-9441-3ec3aa596bf4 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:29.330079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c69f7-0478-4827-a0bc-ce192fafe933 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:29.371692 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08cdba0-ea55-4deb-877f-a03a65d0c179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:29.372275 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cdba0-ea55-4deb-877f-a03a65d0c179 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:29.372555 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cdba0-ea55-4deb-877f-a03a65d0c179 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:29.372829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cdba0-ea55-4deb-877f-a03a65d0c179 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:29.387828 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08cdba0-ea55-4deb-877f-a03a65d0c179 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:29.392925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-c08cdba0-ea55-4deb-877f-a03a65d0c179 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:19:29.392925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:29.392925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:29.392925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:29.392925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:29.392925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:29.392925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:29.392925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:29.392925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:29.392925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:29.392925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:29.392925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:29.392925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:29.392925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:29.392925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:29.392925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:29.393872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:29.393872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:29.393872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:29.393872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:29.393872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:29.393872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:29.393872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:29.393872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:29.393872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:29.393872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:29.393872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:29.393872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:29.393872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:29.393872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:29.393872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:29.393872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:29.394950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:29.394950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:29.394950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:29.394950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:29.394950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:29.394950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:29.394950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:29.394950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:29.394950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:29.394950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:29.394950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:19:29.394950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:29.394950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 156/302] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:29.395817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d7aac-7c75-4fef-9e93-f8fd7c20a01d None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:29.396016 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d7aac-7c75-4fef-9e93-f8fd7c20a01d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:29.396235 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d7aac-7c75-4fef-9e93-f8fd7c20a01d None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:29.397531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 157/303] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:29.401397 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d7aac-7c75-4fef-9e93-f8fd7c20a01d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:29.415182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf9c6f-9dbb-4ae2-9441-3ec3aa596bf4 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:29.435261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:29.459571 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9b306-0b69-496d-8e17-bc4b40e396ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:29.460308 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9b306-0b69-496d-8e17-bc4b40e396ca None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:29.460593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9b306-0b69-496d-8e17-bc4b40e396ca None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:29.460861 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9b306-0b69-496d-8e17-bc4b40e396ca None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:29.462362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9b306-0b69-496d-8e17-bc4b40e396ca None admin] RBAC: Adding query filter params (name=member) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:29.462362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9b306-0b69-496d-8e17-bc4b40e396ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:29.463775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9b306-0b69-496d-8e17-bc4b40e396ca None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:29.473584 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 147/304] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:29.479190 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9b306-0b69-496d-8e17-bc4b40e396ca None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:29.523035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d7aac-7c75-4fef-9e93-f8fd7c20a01d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:29.524779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:29.558876 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e9af7c-c29e-4c7d-be6e-c02096ce45cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:29.559514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9af7c-c29e-4c7d-be6e-c02096ce45cb None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:29.559795 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9af7c-c29e-4c7d-be6e-c02096ce45cb None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:29.560105 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e9af7c-c29e-4c7d-be6e-c02096ce45cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:29.561215 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9af7c-c29e-4c7d-be6e-c02096ce45cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:29.562676 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e9af7c-c29e-4c7d-be6e-c02096ce45cb None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:29.564280 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 158/305] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:19:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:29.565396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7716f9f-6d51-4705-98b0-b69da4dd5fa9 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:29.565704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7716f9f-6d51-4705-98b0-b69da4dd5fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:29.565982 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7716f9f-6d51-4705-98b0-b69da4dd5fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:29.585014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b7716f9f-6d51-4705-98b0-b69da4dd5fa9 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:29.586208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b7716f9f-6d51-4705-98b0-b69da4dd5fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:29.588623 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9b306-0b69-496d-8e17-bc4b40e396ca None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:29.622768 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee1c98-ab84-4d73-9c79-e0229940ebc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:29.623293 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee1c98-ab84-4d73-9c79-e0229940ebc7 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:29.623642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee1c98-ab84-4d73-9c79-e0229940ebc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:29.623919 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee1c98-ab84-4d73-9c79-e0229940ebc7 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:29.624623 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee1c98-ab84-4d73-9c79-e0229940ebc7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:29.624929 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee1c98-ab84-4d73-9c79-e0229940ebc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:29.626324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee1c98-ab84-4d73-9c79-e0229940ebc7 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:29.635843 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 148/306] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:29.636796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee1c98-ab84-4d73-9c79-e0229940ebc7 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:29.669063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 159/307] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:29.673313 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7716f9f-6d51-4705-98b0-b69da4dd5fa9 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:29.699443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:29.751458 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee1c98-ab84-4d73-9c79-e0229940ebc7 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:29.753371 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:29.788571 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7716f9f-6d51-4705-98b0-b69da4dd5fa9 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:29.790391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b40804-c587-4852-b7be-7f4c5b9b662c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:29.791052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b40804-c587-4852-b7be-7f4c5b9b662c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:29.791335 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b40804-c587-4852-b7be-7f4c5b9b662c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:29.791621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b40804-c587-4852-b7be-7f4c5b9b662c None admin] PATH_INFO: `/v3/users/2abdbfef38d147b3805cd5be2e10783d` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:29.792787 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b40804-c587-4852-b7be-7f4c5b9b662c None admin] RBAC: Authorizing `identity:get_user(user_id=2abdbfef38d147b3805cd5be2e10783d)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:29.794244 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b40804-c587-4852-b7be-7f4c5b9b662c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:29.795811 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 149/308] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:29 2022] GET /identity/v3/users/2abdbfef38d147b3805cd5be2e10783d => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:29.796605 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b40804-c587-4852-b7be-7f4c5b9b662c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:29.830430 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18618e1e-5a54-4bfd-92c8-462042bc538c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:29.831051 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18618e1e-5a54-4bfd-92c8-462042bc538c None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:29.831328 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18618e1e-5a54-4bfd-92c8-462042bc538c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:29.831623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18618e1e-5a54-4bfd-92c8-462042bc538c None admin] PATH_INFO: `/v3/groups` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:29.832251 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18618e1e-5a54-4bfd-92c8-462042bc538c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:29.833766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18618e1e-5a54-4bfd-92c8-462042bc538c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:29.849381 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 160/309] 10.222.0.18 () {60 vars in 1134 bytes} [Mon Aug 29 20:19:29 2022] POST /identity/v3/groups => generated 229 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:19:29.852111 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18618e1e-5a54-4bfd-92c8-462042bc538c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:29.956763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b40804-c587-4852-b7be-7f4c5b9b662c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:29.958641 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:29.981812 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18618e1e-5a54-4bfd-92c8-462042bc538c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:30.007410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46877518-0e8c-49e6-add1-a33f1b28ca52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:30.008079 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46877518-0e8c-49e6-add1-a33f1b28ca52 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:30.008363 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46877518-0e8c-49e6-add1-a33f1b28ca52 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:30.008636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46877518-0e8c-49e6-add1-a33f1b28ca52 None admin] PATH_INFO: `/v3/users/f263250a762a40d4a6c2b8584298af35` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:30.024059 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46877518-0e8c-49e6-add1-a33f1b28ca52 None admin] RBAC: Authorizing `identity:get_user(user_id=f263250a762a40d4a6c2b8584298af35)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:30.024441 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49c56f-e50a-498c-9959-cb9c997eaf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:30.025688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49c56f-e50a-498c-9959-cb9c997eaf55 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:30.025970 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49c56f-e50a-498c-9959-cb9c997eaf55 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:30.026070 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46877518-0e8c-49e6-add1-a33f1b28ca52 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:30.026236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49c56f-e50a-498c-9959-cb9c997eaf55 None admin] PATH_INFO: `/v3/system/users/2abdbfef38d147b3805cd5be2e10783d/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:30.027752 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd49c56f-e50a-498c-9959-cb9c997eaf55 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2abdbfef38d147b3805cd5be2e10783d, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:30.028867 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 150/310] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:29 2022] GET /identity/v3/users/f263250a762a40d4a6c2b8584298af35 => generated 293 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:30.029176 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd49c56f-e50a-498c-9959-cb9c997eaf55 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:30.029511 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46877518-0e8c-49e6-add1-a33f1b28ca52 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:30.041071 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 161/311] 10.222.0.18 () {58 vars in 1334 bytes} [Mon Aug 29 20:19:29 2022] PUT /identity/v3/system/users/2abdbfef38d147b3805cd5be2e10783d/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:19:30.041664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49c56f-e50a-498c-9959-cb9c997eaf55 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:30.064213 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:30.166111 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46877518-0e8c-49e6-add1-a33f1b28ca52 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:30.185730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49c56f-e50a-498c-9959-cb9c997eaf55 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:30.208040 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00a590-037a-4e1c-98b3-a8cdd4a68c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:30.208629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00a590-037a-4e1c-98b3-a8cdd4a68c62 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:30.208963 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00a590-037a-4e1c-98b3-a8cdd4a68c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:30.209246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00a590-037a-4e1c-98b3-a8cdd4a68c62 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:30.223869 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00a590-037a-4e1c-98b3-a8cdd4a68c62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:30.227658 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db94fc8-09ee-40ab-b308-194f0e30daa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:30.228318 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db94fc8-09ee-40ab-b308-194f0e30daa8 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:30.228604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db94fc8-09ee-40ab-b308-194f0e30daa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:30.228885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db94fc8-09ee-40ab-b308-194f0e30daa8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:30.228983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-4b00a590-037a-4e1c-98b3-a8cdd4a68c62 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:19:30.228983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:30.228983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:30.228983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:30.228983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:30.228983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:30.228983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:30.228983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:30.228983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:30.228983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:30.228983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:30.228983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:30.228983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:30.228983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:30.228983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:30.228983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:30.229796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:30.229796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:30.229796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:30.229796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:30.229796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:30.229796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:30.229796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:30.229796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:30.229796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:30.229796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:30.229796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:30.229796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:30.229796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:30.229796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:30.229796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:30.229796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:30.230885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:30.230885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:30.230885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:30.230885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:30.230885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:30.230885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:30.230885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:30.230885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:30.230885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:30.230885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:30.230885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:19:30.230885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:30.231526 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db94fc8-09ee-40ab-b308-194f0e30daa8 None admin] RBAC: Adding query filter params (role.id=5ae3132138594bb7a1fbacc9d93198c1, user.id=f263250a762a40d4a6c2b8584298af35, scope.system=all) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:30.231526 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db94fc8-09ee-40ab-b308-194f0e30daa8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:30.231655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 151/312] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:30.231717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db94fc8-09ee-40ab-b308-194f0e30daa8 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:30.238910 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00a590-037a-4e1c-98b3-a8cdd4a68c62 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:30.242445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 162/313] 10.222.0.18 () {56 vars in 1308 bytes} [Mon Aug 29 20:19:30 2022] GET /identity/v3/role_assignments?user.id=f263250a762a40d4a6c2b8584298af35&scope.system=all&role.id=5ae3132138594bb7a1fbacc9d93198c1 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:30.348348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:30.360820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00a590-037a-4e1c-98b3-a8cdd4a68c62 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:30.401157 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbff99b-effd-4175-9c31-9ef2ab7c0834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:30.401849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbff99b-effd-4175-9c31-9ef2ab7c0834 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:30.402130 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbff99b-effd-4175-9c31-9ef2ab7c0834 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:30.402469 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbff99b-effd-4175-9c31-9ef2ab7c0834 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:30.403153 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbff99b-effd-4175-9c31-9ef2ab7c0834 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:30.403514 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbff99b-effd-4175-9c31-9ef2ab7c0834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:30.404939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbff99b-effd-4175-9c31-9ef2ab7c0834 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:30.414718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 152/314] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:30.420432 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db94fc8-09ee-40ab-b308-194f0e30daa8 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:30.434504 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:30.533046 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db94fc8-09ee-40ab-b308-194f0e30daa8 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:30.570510 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e981c-bb19-4efb-8bea-c837fbde4587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:30.571527 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e981c-bb19-4efb-8bea-c837fbde4587 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:30.572123 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e981c-bb19-4efb-8bea-c837fbde4587 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:30.573081 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e981c-bb19-4efb-8bea-c837fbde4587 None admin] PATH_INFO: `/v3/users/1fb2d08e87d949798c6b8cccc5e1807d` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:30.574456 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e981c-bb19-4efb-8bea-c837fbde4587 None admin] RBAC: Authorizing `identity:get_user(user_id=1fb2d08e87d949798c6b8cccc5e1807d)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:30.576134 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e981c-bb19-4efb-8bea-c837fbde4587 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:30.577946 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 163/315] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:30 2022] GET /identity/v3/users/1fb2d08e87d949798c6b8cccc5e1807d => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:30.584020 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbff99b-effd-4175-9c31-9ef2ab7c0834 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:30.692254 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbff99b-effd-4175-9c31-9ef2ab7c0834 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:30.728669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adf0a4-9852-45ee-9159-391c188fe7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:30.729284 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adf0a4-9852-45ee-9159-391c188fe7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:30.729559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adf0a4-9852-45ee-9159-391c188fe7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:30.729828 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adf0a4-9852-45ee-9159-391c188fe7c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:30.749924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:30.803584 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9adf0a4-9852-45ee-9159-391c188fe7c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:30.804992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9adf0a4-9852-45ee-9159-391c188fe7c5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:30.912146 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 153/316] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:30.913986 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e981c-bb19-4efb-8bea-c837fbde4587 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:31.000743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:31.024845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e981c-bb19-4efb-8bea-c837fbde4587 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:31.062093 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14856368-c07c-4845-8541-f1d330887db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:31.063123 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14856368-c07c-4845-8541-f1d330887db6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:31.063636 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14856368-c07c-4845-8541-f1d330887db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:31.064137 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14856368-c07c-4845-8541-f1d330887db6 None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:31.065460 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14856368-c07c-4845-8541-f1d330887db6 None admin] RBAC: Authorizing `identity:get_project(project_id=eddee91f1cf44ccf83ca7cd9419ab306)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:31.067106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14856368-c07c-4845-8541-f1d330887db6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:31.068972 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 164/317] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:30 2022] GET /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:31.078674 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adf0a4-9852-45ee-9159-391c188fe7c5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:31.163975 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:31.216711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adf0a4-9852-45ee-9159-391c188fe7c5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:31.264422 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1710df-4af1-4284-961b-5da0815555d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:31.265053 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1710df-4af1-4284-961b-5da0815555d3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:31.265330 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1710df-4af1-4284-961b-5da0815555d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:31.265597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1710df-4af1-4284-961b-5da0815555d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:31.266353 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1710df-4af1-4284-961b-5da0815555d3 None admin] RBAC: Adding query filter params (role.id=5ae3132138594bb7a1fbacc9d93198c1, scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306, user.id=1fb2d08e87d949798c6b8cccc5e1807d) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:31.266721 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1710df-4af1-4284-961b-5da0815555d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:31.268322 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1710df-4af1-4284-961b-5da0815555d3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:31.275524 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 154/318] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:19:31 2022] GET /identity/v3/role_assignments?user.id=1fb2d08e87d949798c6b8cccc5e1807d&scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306&role.id=5ae3132138594bb7a1fbacc9d93198c1 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:31.998924 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7789a-f2c9-4145-89fb-ddf8eab74669 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:31.999440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7789a-f2c9-4145-89fb-ddf8eab74669 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:31.999515 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7789a-f2c9-4145-89fb-ddf8eab74669 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:32.000997 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 165/319] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:19:32.006086 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e6aa9-c7ee-4a9b-8d97-135d869ca15a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:32.006372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e6aa9-c7ee-4a9b-8d97-135d869ca15a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:32.006650 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e6aa9-c7ee-4a9b-8d97-135d869ca15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:32.025919 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-677e6aa9-c7ee-4a9b-8d97-135d869ca15a None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:32.027121 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-677e6aa9-c7ee-4a9b-8d97-135d869ca15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:32.098613 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 155/320] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:32.117834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c050613-d398-4bf9-b4a9-35ddd569ce98 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:32.118142 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c050613-d398-4bf9-b4a9-35ddd569ce98 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:32.118388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c050613-d398-4bf9-b4a9-35ddd569ce98 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:32.120320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 166/321] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:32.125698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42494188-d8da-4ba4-bd93-4ff6beb0db2a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:32.125979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42494188-d8da-4ba4-bd93-4ff6beb0db2a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:32.126245 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42494188-d8da-4ba4-bd93-4ff6beb0db2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:32.145459 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-42494188-d8da-4ba4-bd93-4ff6beb0db2a None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:32.146634 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-42494188-d8da-4ba4-bd93-4ff6beb0db2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:32.190728 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:32.222995 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 156/322] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:32.231794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca613f-6769-4158-bef1-3c13afc0c00e None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:32.232116 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca613f-6769-4158-bef1-3c13afc0c00e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:32.232397 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca613f-6769-4158-bef1-3c13afc0c00e None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:32.233859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 167/323] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:32.238867 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66869a-87ff-471c-9334-d4d01d6208a2 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:32.239152 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66869a-87ff-471c-9334-d4d01d6208a2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:32.239426 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66869a-87ff-471c-9334-d4d01d6208a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:32.258227 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-be66869a-87ff-471c-9334-d4d01d6208a2 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:32.259332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-be66869a-87ff-471c-9334-d4d01d6208a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:32.337521 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 157/324] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:32.371019 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca613f-6769-4158-bef1-3c13afc0c00e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:32.478109 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca613f-6769-4158-bef1-3c13afc0c00e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:32.490323 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66869a-87ff-471c-9334-d4d01d6208a2 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:32.517220 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dd251-5d67-4a82-ad16-421b762d2d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:32.517765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dd251-5d67-4a82-ad16-421b762d2d2a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:32.518043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dd251-5d67-4a82-ad16-421b762d2d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:32.518315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dd251-5d67-4a82-ad16-421b762d2d2a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:32.519423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65dd251-5d67-4a82-ad16-421b762d2d2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:32.520867 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65dd251-5d67-4a82-ad16-421b762d2d2a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:32.522424 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 168/325] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:19:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:32.530077 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dd251-5d67-4a82-ad16-421b762d2d2a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:32.550688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:32.607224 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66869a-87ff-471c-9334-d4d01d6208a2 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:32.609929 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:32.652086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dd251-5d67-4a82-ad16-421b762d2d2a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:32.665884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20342298-1e33-4b66-8b57-bee3d951c10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:32.666602 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20342298-1e33-4b66-8b57-bee3d951c10c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:32.666873 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20342298-1e33-4b66-8b57-bee3d951c10c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:32.667139 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20342298-1e33-4b66-8b57-bee3d951c10c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:32.677150 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20342298-1e33-4b66-8b57-bee3d951c10c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:32.681971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-20342298-1e33-4b66-8b57-bee3d951c10c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:32.681971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:32.681971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:32.681971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:32.681971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:32.681971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:32.681971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:32.681971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:32.681971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:32.681971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:32.681971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:32.681971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:32.681971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:32.681971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:32.681971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:32.681971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:32.683072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:32.683072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:32.683072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:32.683072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:32.683072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:32.683072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:32.683072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:32.683072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:32.683072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:32.683072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:32.683072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:32.683072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:32.683072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:32.683072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:32.683072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:32.683072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:32.684095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:32.684095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:32.684095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:32.684095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:32.684095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:32.684095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:32.684095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:32.684095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:32.684095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:32.684095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:32.684095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:32.684095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:32.684095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 158/326] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:19:32.688945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20342298-1e33-4b66-8b57-bee3d951c10c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:32.689420 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83a9d2-6762-4265-816a-33dc741a2f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:32.690040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83a9d2-6762-4265-816a-33dc741a2f90 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:32.690325 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83a9d2-6762-4265-816a-33dc741a2f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:32.690600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83a9d2-6762-4265-816a-33dc741a2f90 None admin] PATH_INFO: `/v3/groups` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:32.691246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d83a9d2-6762-4265-816a-33dc741a2f90 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:32.692720 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d83a9d2-6762-4265-816a-33dc741a2f90 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:32.707637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 169/327] 10.222.0.18 () {60 vars in 1134 bytes} [Mon Aug 29 20:19:32 2022] POST /identity/v3/groups => generated 226 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:19:32.708499 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83a9d2-6762-4265-816a-33dc741a2f90 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:32.806497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:32.819265 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20342298-1e33-4b66-8b57-bee3d951c10c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:32.841677 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83a9d2-6762-4265-816a-33dc741a2f90 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:32.867528 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d4cd8-c53c-4746-8ecc-5c5925b78914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:32.869006 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d4cd8-c53c-4746-8ecc-5c5925b78914 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:32.869006 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d4cd8-c53c-4746-8ecc-5c5925b78914 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:32.869006 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d4cd8-c53c-4746-8ecc-5c5925b78914 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:32.880196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d4cd8-c53c-4746-8ecc-5c5925b78914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:32.885802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-cd8d4cd8-c53c-4746-8ecc-5c5925b78914 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:19:32.885802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:32.885802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:32.885802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:32.885802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:32.885802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:32.885802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:32.885802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:32.885802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:32.885802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:32.885802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:32.885802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:32.885802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:32.885802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:32.885802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:32.885802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:32.886862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:32.886862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:32.886862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:32.886862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:32.886862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:32.886862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:32.886862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:32.886862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:32.886862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:32.886862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:32.886862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:32.886862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:32.886862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:32.886862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:32.886862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:32.886862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:32.887666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:32.887666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:32.887666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:32.887666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:32.887666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:32.887666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:32.887666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:32.887666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:32.887666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:32.887666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:32.887666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:19:32.887666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:32.887666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 159/328] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:32.896237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d4cd8-c53c-4746-8ecc-5c5925b78914 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:32.897146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1878ed-275a-48bc-90eb-9c734636fb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:32.897774 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1878ed-275a-48bc-90eb-9c734636fb5d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:32.898054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1878ed-275a-48bc-90eb-9c734636fb5d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:32.898367 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1878ed-275a-48bc-90eb-9c734636fb5d None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:32.899051 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1878ed-275a-48bc-90eb-9c734636fb5d None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:32.899561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1878ed-275a-48bc-90eb-9c734636fb5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:32.900834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1878ed-275a-48bc-90eb-9c734636fb5d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:32.911014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 170/329] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:32.916475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1878ed-275a-48bc-90eb-9c734636fb5d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:32.946599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:33.051183 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d4cd8-c53c-4746-8ecc-5c5925b78914 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:33.059324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1878ed-275a-48bc-90eb-9c734636fb5d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:33.103407 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ee3d0-4827-48f1-9e6d-adb46d558cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:33.104510 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220123be-9630-45b3-b060-00bb4937edfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:33.104604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ee3d0-4827-48f1-9e6d-adb46d558cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:33.104860 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ee3d0-4827-48f1-9e6d-adb46d558cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:33.105581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220123be-9630-45b3-b060-00bb4937edfe None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:33.105702 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ee3d0-4827-48f1-9e6d-adb46d558cc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:33.105778 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220123be-9630-45b3-b060-00bb4937edfe None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:33.105844 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ee3d0-4827-48f1-9e6d-adb46d558cc2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:33.105908 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220123be-9630-45b3-b060-00bb4937edfe None admin] PATH_INFO: `/v3/users/2abdbfef38d147b3805cd5be2e10783d` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:33.106255 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ee3d0-4827-48f1-9e6d-adb46d558cc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:33.107143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220123be-9630-45b3-b060-00bb4937edfe None admin] RBAC: Authorizing `identity:get_user(user_id=2abdbfef38d147b3805cd5be2e10783d)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:33.107659 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ee3d0-4827-48f1-9e6d-adb46d558cc2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:33.108654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220123be-9630-45b3-b060-00bb4937edfe None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:33.110715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 171/330] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:32 2022] GET /identity/v3/users/2abdbfef38d147b3805cd5be2e10783d => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:33.116636 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220123be-9630-45b3-b060-00bb4937edfe None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:33.120677 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 160/331] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:33.121193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:33.127620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ee3d0-4827-48f1-9e6d-adb46d558cc2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:33.240826 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:33.261485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220123be-9630-45b3-b060-00bb4937edfe None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:33.262066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ee3d0-4827-48f1-9e6d-adb46d558cc2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:33.305970 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386d9bf-c13c-4c5e-a22e-8c01a37abe97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:33.306882 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386d9bf-c13c-4c5e-a22e-8c01a37abe97 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:33.307805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386d9bf-c13c-4c5e-a22e-8c01a37abe97 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:33.310132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386d9bf-c13c-4c5e-a22e-8c01a37abe97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:33.310585 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386d9bf-c13c-4c5e-a22e-8c01a37abe97 None admin] RBAC: Adding query filter params (role.id=89cf5ed93cc642588a54639342c59ea6, user.id=2abdbfef38d147b3805cd5be2e10783d, scope.system=all) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:33.311059 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386d9bf-c13c-4c5e-a22e-8c01a37abe97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:33.313378 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5ed83-224e-4512-9835-4f31960de0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:33.314173 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386d9bf-c13c-4c5e-a22e-8c01a37abe97 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:33.319976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5ed83-224e-4512-9835-4f31960de0c5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:33.320400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5ed83-224e-4512-9835-4f31960de0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:33.320691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5ed83-224e-4512-9835-4f31960de0c5 None admin] PATH_INFO: `/v3/users/f263250a762a40d4a6c2b8584298af35` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:33.324682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5ed83-224e-4512-9835-4f31960de0c5 None admin] RBAC: Authorizing `identity:get_user(user_id=f263250a762a40d4a6c2b8584298af35)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:33.326247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5ed83-224e-4512-9835-4f31960de0c5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:33.328214 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 161/332] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:33 2022] GET /identity/v3/users/f263250a762a40d4a6c2b8584298af35 => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:33.335603 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 172/333] 10.222.0.18 () {56 vars in 1308 bytes} [Mon Aug 29 20:19:33 2022] GET /identity/v3/role_assignments?user.id=2abdbfef38d147b3805cd5be2e10783d&scope.system=all&role.id=89cf5ed93cc642588a54639342c59ea6 => generated 510 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:33.338226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5ed83-224e-4512-9835-4f31960de0c5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:33.478250 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5ed83-224e-4512-9835-4f31960de0c5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:33.524658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a2371-ffba-484b-aa46-636846a245e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:33.526302 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a2371-ffba-484b-aa46-636846a245e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:33.526774 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a2371-ffba-484b-aa46-636846a245e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:33.527238 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a2371-ffba-484b-aa46-636846a245e8 None admin] PATH_INFO: `/v3/system/users/f263250a762a40d4a6c2b8584298af35/roles/5ae3132138594bb7a1fbacc9d93198c1` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:33.529730 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a2371-ffba-484b-aa46-636846a245e8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f263250a762a40d4a6c2b8584298af35, role_id=5ae3132138594bb7a1fbacc9d93198c1)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:33.531273 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a2371-ffba-484b-aa46-636846a245e8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:33.544843 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 162/334] 10.222.0.18 () {58 vars in 1334 bytes} [Mon Aug 29 20:19:33 2022] PUT /identity/v3/system/users/f263250a762a40d4a6c2b8584298af35/roles/5ae3132138594bb7a1fbacc9d93198c1 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:19:34.926600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc421c-5abc-4908-9abf-c20327bab38c None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:34.927119 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc421c-5abc-4908-9abf-c20327bab38c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:34.927194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc421c-5abc-4908-9abf-c20327bab38c None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:34.928639 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 173/335] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:34.935139 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0c607-8d8b-48bc-b8f9-f8710f9e6879 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:34.935603 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0c607-8d8b-48bc-b8f9-f8710f9e6879 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:34.935678 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0c607-8d8b-48bc-b8f9-f8710f9e6879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:34.957989 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-20c0c607-8d8b-48bc-b8f9-f8710f9e6879 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:34.959207 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-20c0c607-8d8b-48bc-b8f9-f8710f9e6879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:34.974319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:35.039238 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 163/336] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:35.417335 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5fae7-370a-4ff9-be0f-a3616f747c90 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:35.417623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5fae7-370a-4ff9-be0f-a3616f747c90 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:35.418222 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5fae7-370a-4ff9-be0f-a3616f747c90 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:35.420105 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 174/337] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:35.425847 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b5060-3fbe-4d23-9eff-d59c7028316a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:35.426489 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b5060-3fbe-4d23-9eff-d59c7028316a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:35.426966 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b5060-3fbe-4d23-9eff-d59c7028316a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:35.457139 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-972b5060-3fbe-4d23-9eff-d59c7028316a None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:35.458712 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-972b5060-3fbe-4d23-9eff-d59c7028316a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:35.524031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5fae7-370a-4ff9-be0f-a3616f747c90 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:35.537360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 164/338] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:35.624665 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:35.677752 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5fae7-370a-4ff9-be0f-a3616f747c90 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:35.732425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65ead8-17c7-42bc-8d6d-3b3c35e645e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:35.733086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65ead8-17c7-42bc-8d6d-3b3c35e645e4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:35.733367 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65ead8-17c7-42bc-8d6d-3b3c35e645e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:35.733642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65ead8-17c7-42bc-8d6d-3b3c35e645e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:35.750127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b65ead8-17c7-42bc-8d6d-3b3c35e645e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:35.755797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-5b65ead8-17c7-42bc-8d6d-3b3c35e645e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:35.755797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:35.755797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:35.755797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:35.755797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:35.755797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:35.755797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:35.755797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:35.755797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:35.755797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:35.755797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:35.755797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:35.755797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:35.755797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:35.755797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:35.755797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:35.757359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:35.757359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:35.757359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:35.757359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:35.757359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:35.757359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:35.757359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:35.757359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:35.757359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:35.757359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:35.757359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:35.757359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:35.757359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:35.757359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:35.757359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:35.757359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:35.758141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:35.758141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:35.758141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:35.758141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:35.758141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:35.758141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:35.758141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:35.758141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:35.758141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:35.758141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:35.758141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:35.758141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:35.762412 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 175/339] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:35.765075 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b5060-3fbe-4d23-9eff-d59c7028316a None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:35.885248 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba2b84-2543-42a5-9dfc-27e245c8101c None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:35.886077 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba2b84-2543-42a5-9dfc-27e245c8101c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:35.886290 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cba2b84-2543-42a5-9dfc-27e245c8101c None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:35.891193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 176/340] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:35.892903 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:35.902809 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec16cf0-4522-41ae-a23e-ba9df0bf5405 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:35.903218 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec16cf0-4522-41ae-a23e-ba9df0bf5405 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:35.903614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec16cf0-4522-41ae-a23e-ba9df0bf5405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:35.926265 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b5060-3fbe-4d23-9eff-d59c7028316a None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:35.949429 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-aec16cf0-4522-41ae-a23e-ba9df0bf5405 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:35.950717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-aec16cf0-4522-41ae-a23e-ba9df0bf5405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:36.002757 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8a31c-a0d2-4b06-ad82-87e596d715d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:36.004141 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8a31c-a0d2-4b06-ad82-87e596d715d1 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:36.004625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8a31c-a0d2-4b06-ad82-87e596d715d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:36.005162 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8a31c-a0d2-4b06-ad82-87e596d715d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:36.006071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8a31c-a0d2-4b06-ad82-87e596d715d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:36.006611 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8a31c-a0d2-4b06-ad82-87e596d715d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:36.008312 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8a31c-a0d2-4b06-ad82-87e596d715d1 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:36.027773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 165/341] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:36.034107 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8a31c-a0d2-4b06-ad82-87e596d715d1 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:36.150608 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 177/342] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:36.152105 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec16cf0-4522-41ae-a23e-ba9df0bf5405 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:36.225409 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8a31c-a0d2-4b06-ad82-87e596d715d1 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:36.275959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:36.298119 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fb6a4-cb97-47c9-844e-1e48f1ba53c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:36.298751 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fb6a4-cb97-47c9-844e-1e48f1ba53c4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:36.299030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fb6a4-cb97-47c9-844e-1e48f1ba53c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:36.299305 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fb6a4-cb97-47c9-844e-1e48f1ba53c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:36.316341 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec16cf0-4522-41ae-a23e-ba9df0bf5405 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:36.317634 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:36.392261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64714cf9-4e89-4b42-a23f-c15977827a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:36.393097 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64714cf9-4e89-4b42-a23f-c15977827a69 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:36.393481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64714cf9-4e89-4b42-a23f-c15977827a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:36.393881 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64714cf9-4e89-4b42-a23f-c15977827a69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:36.415524 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fb6a4-cb97-47c9-844e-1e48f1ba53c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:36.417506 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fb6a4-cb97-47c9-844e-1e48f1ba53c4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:36.419578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64714cf9-4e89-4b42-a23f-c15977827a69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:36.434782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-64714cf9-4e89-4b42-a23f-c15977827a69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:36.434782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:36.434782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:36.434782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:36.434782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:36.434782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:36.434782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:36.434782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:36.434782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:36.434782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:36.434782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:36.434782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:36.434782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:36.434782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:36.434782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:36.434782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:36.435709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:36.435709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:36.435709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:36.435709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:36.435709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:36.435709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:36.435709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:36.435709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:36.435709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:36.435709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:36.435709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:36.435709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:36.435709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:36.435709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:36.435709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:36.435709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:36.436529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:36.436529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:36.436529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:36.436529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:36.436529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:36.436529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:36.436529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:36.436529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:36.436529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:36.436529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:36.436529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:36.436529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:36.436529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 178/343] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:19:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:36.443256 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64714cf9-4e89-4b42-a23f-c15977827a69 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:36.568102 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 166/344] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 20:19:36.570682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fb6a4-cb97-47c9-844e-1e48f1ba53c4 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:36.612518 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64714cf9-4e89-4b42-a23f-c15977827a69 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:36.616634 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:36.640398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:36.659672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f3b2b-5a37-4aa0-8e84-a18195de4f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:36.660334 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f3b2b-5a37-4aa0-8e84-a18195de4f9e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:36.660616 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f3b2b-5a37-4aa0-8e84-a18195de4f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:36.661146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f3b2b-5a37-4aa0-8e84-a18195de4f9e None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:36.661829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f3b2b-5a37-4aa0-8e84-a18195de4f9e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:36.662132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f3b2b-5a37-4aa0-8e84-a18195de4f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:36.663532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f3b2b-5a37-4aa0-8e84-a18195de4f9e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:36.676109 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 179/345] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:19:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:36.677792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97306e7-1e0e-468f-bdb9-29ca2a0e616c None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:36.678287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97306e7-1e0e-468f-bdb9-29ca2a0e616c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:36.678772 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97306e7-1e0e-468f-bdb9-29ca2a0e616c None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:36.680375 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 180/346] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:36.681384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97306e7-1e0e-468f-bdb9-29ca2a0e616c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:36.685552 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fb6a4-cb97-47c9-844e-1e48f1ba53c4 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:36.732168 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d161ac4-18c9-4767-a192-18617345190d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:36.733828 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d161ac4-18c9-4767-a192-18617345190d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:36.734394 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d161ac4-18c9-4767-a192-18617345190d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:36.734886 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d161ac4-18c9-4767-a192-18617345190d None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:36.736581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d161ac4-18c9-4767-a192-18617345190d None admin] RBAC: Authorizing `identity:get_project(project_id=0370c388ce4e42d2ad10ebca467e24dc)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:36.739122 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d161ac4-18c9-4767-a192-18617345190d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:36.742733 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 167/347] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:36 2022] GET /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:36.744025 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d161ac4-18c9-4767-a192-18617345190d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:36.822567 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97306e7-1e0e-468f-bdb9-29ca2a0e616c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:36.869793 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf977db-f587-4f2a-adcd-54117062b9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:36.870788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf977db-f587-4f2a-adcd-54117062b9f6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:36.871283 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf977db-f587-4f2a-adcd-54117062b9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:36.872955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf977db-f587-4f2a-adcd-54117062b9f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:36.886462 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d161ac4-18c9-4767-a192-18617345190d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:36.890028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf977db-f587-4f2a-adcd-54117062b9f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:36.895740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-eaf977db-f587-4f2a-adcd-54117062b9f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:36.895740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:36.895740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:36.895740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:36.895740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:36.895740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:36.895740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:36.895740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:36.895740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:36.895740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:36.895740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:36.895740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:36.895740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:36.895740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:36.895740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:36.895740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:36.896717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:36.896717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:36.896717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:36.896717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:36.896717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:36.896717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:36.896717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:36.896717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:36.896717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:36.896717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:36.896717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:36.896717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:36.896717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:36.896717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:36.896717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:36.896717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:36.897514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:36.897514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:36.897514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:36.897514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:36.897514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:36.897514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:36.897514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:36.897514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:36.897514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:36.897514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:36.897514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:36.897514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:36.900666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 181/348] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:36.902252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:36.908269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473debd8-e706-4096-b357-4b76b9b77277 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:36.908557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473debd8-e706-4096-b357-4b76b9b77277 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:36.908847 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473debd8-e706-4096-b357-4b76b9b77277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:36.942358 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-473debd8-e706-4096-b357-4b76b9b77277 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:36.949937 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-473debd8-e706-4096-b357-4b76b9b77277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:36.953808 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ebf7c3-3e29-4096-964d-f7892c9ef86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:36.955641 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ebf7c3-3e29-4096-964d-f7892c9ef86d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:36.960474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ebf7c3-3e29-4096-964d-f7892c9ef86d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:36.961048 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ebf7c3-3e29-4096-964d-f7892c9ef86d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:36.995884 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:37.059603 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 182/349] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:37.060512 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473debd8-e706-4096-b357-4b76b9b77277 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:37.072636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ebf7c3-3e29-4096-964d-f7892c9ef86d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:37.075057 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ebf7c3-3e29-4096-964d-f7892c9ef86d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:37.173560 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473debd8-e706-4096-b357-4b76b9b77277 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:37.194113 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 168/350] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:37.195568 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ebf7c3-3e29-4096-964d-f7892c9ef86d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:37.207786 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:37.212149 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87abb8c1-49cd-4a1b-bcfa-45f13f364f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:37.212735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abb8c1-49cd-4a1b-bcfa-45f13f364f9a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:37.213016 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abb8c1-49cd-4a1b-bcfa-45f13f364f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:37.213290 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87abb8c1-49cd-4a1b-bcfa-45f13f364f9a None admin] PATH_INFO: `/v3/groups/80832b8b4bff430eb00f2ad1f2236c03` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:37.222788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87abb8c1-49cd-4a1b-bcfa-45f13f364f9a None admin] RBAC: Authorizing `identity:get_group(group_id=80832b8b4bff430eb00f2ad1f2236c03)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:37.224396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87abb8c1-49cd-4a1b-bcfa-45f13f364f9a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:37.228496 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 183/351] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 29 20:19:37 2022] GET /identity/v3/groups/80832b8b4bff430eb00f2ad1f2236c03 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:37.228496 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87abb8c1-49cd-4a1b-bcfa-45f13f364f9a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:37.261963 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:37.334439 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ebf7c3-3e29-4096-964d-f7892c9ef86d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:37.373016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e818f-2f00-4b6e-b8ab-484ae46ca436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:37.374021 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e818f-2f00-4b6e-b8ab-484ae46ca436 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:37.374568 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e818f-2f00-4b6e-b8ab-484ae46ca436 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:37.375037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e818f-2f00-4b6e-b8ab-484ae46ca436 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:37.376105 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e818f-2f00-4b6e-b8ab-484ae46ca436 None admin] RBAC: Adding query filter params (name=member) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:37.376616 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e818f-2f00-4b6e-b8ab-484ae46ca436 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:37.378377 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e818f-2f00-4b6e-b8ab-484ae46ca436 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:37.383205 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87abb8c1-49cd-4a1b-bcfa-45f13f364f9a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:37.389661 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 169/352] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:37.390956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e818f-2f00-4b6e-b8ab-484ae46ca436 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:37.434509 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e440ecb-c98c-446a-a4cf-cfc00f7e85c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:37.435185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e440ecb-c98c-446a-a4cf-cfc00f7e85c7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:37.435470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e440ecb-c98c-446a-a4cf-cfc00f7e85c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:37.435749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e440ecb-c98c-446a-a4cf-cfc00f7e85c7 None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:37.436955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e440ecb-c98c-446a-a4cf-cfc00f7e85c7 None admin] RBAC: Authorizing `identity:get_project(project_id=0370c388ce4e42d2ad10ebca467e24dc)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:37.438471 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e440ecb-c98c-446a-a4cf-cfc00f7e85c7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:37.441816 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 184/353] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:37 2022] GET /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:37.442681 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e440ecb-c98c-446a-a4cf-cfc00f7e85c7 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:37.526001 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:37.554104 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:37.561031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e440ecb-c98c-446a-a4cf-cfc00f7e85c7 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:37.574629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e818f-2f00-4b6e-b8ab-484ae46ca436 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:37.597529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0335fac-7847-4e4b-b8ec-8bcb9bded122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:37.598607 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0335fac-7847-4e4b-b8ec-8bcb9bded122 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:37.598607 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0335fac-7847-4e4b-b8ec-8bcb9bded122 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:37.598800 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0335fac-7847-4e4b-b8ec-8bcb9bded122 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:37.614213 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b8a7f-fcab-4df6-bf12-e436f9f4a592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:37.615339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b8a7f-fcab-4df6-bf12-e436f9f4a592 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:37.615884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b8a7f-fcab-4df6-bf12-e436f9f4a592 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:37.616353 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b8a7f-fcab-4df6-bf12-e436f9f4a592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:37.617091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b8a7f-fcab-4df6-bf12-e436f9f4a592 None admin] RBAC: Adding query filter params (group.id=80832b8b4bff430eb00f2ad1f2236c03, role.id=89cf5ed93cc642588a54639342c59ea6, scope.project.id=0370c388ce4e42d2ad10ebca467e24dc) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:37.617396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b8a7f-fcab-4df6-bf12-e436f9f4a592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:37.619235 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b8a7f-fcab-4df6-bf12-e436f9f4a592 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:37.632063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 170/354] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Aug 29 20:19:37 2022] GET /identity/v3/role_assignments?group.id=80832b8b4bff430eb00f2ad1f2236c03&scope.project.id=0370c388ce4e42d2ad10ebca467e24dc&role.id=89cf5ed93cc642588a54639342c59ea6 => generated 260 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:37.633667 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b8a7f-fcab-4df6-bf12-e436f9f4a592 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:37.685691 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0335fac-7847-4e4b-b8ec-8bcb9bded122 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:37.687513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0335fac-7847-4e4b-b8ec-8bcb9bded122 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:37.787157 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b8a7f-fcab-4df6-bf12-e436f9f4a592 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:37.814432 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:37.829181 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 185/355] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:37.834670 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0335fac-7847-4e4b-b8ec-8bcb9bded122 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:37.852183 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220b3e1-7002-47cf-84e4-7b5b979222e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:37.852847 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220b3e1-7002-47cf-84e4-7b5b979222e6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:37.855884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220b3e1-7002-47cf-84e4-7b5b979222e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:37.855884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220b3e1-7002-47cf-84e4-7b5b979222e6 None admin] PATH_INFO: `/v3/groups/80832b8b4bff430eb00f2ad1f2236c03` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:37.856596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220b3e1-7002-47cf-84e4-7b5b979222e6 None admin] RBAC: Authorizing `identity:get_group(group_id=80832b8b4bff430eb00f2ad1f2236c03)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:37.858052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220b3e1-7002-47cf-84e4-7b5b979222e6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:37.860584 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 171/356] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 29 20:19:37 2022] GET /identity/v3/groups/80832b8b4bff430eb00f2ad1f2236c03 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:37.861767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee33a0f-5185-455d-9e26-e534137b1cd5 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:37.862050 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee33a0f-5185-455d-9e26-e534137b1cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:37.862326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee33a0f-5185-455d-9e26-e534137b1cd5 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:37.868837 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 172/357] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:37.870639 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee33a0f-5185-455d-9e26-e534137b1cd5 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:37.877201 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:37.971981 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0335fac-7847-4e4b-b8ec-8bcb9bded122 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:38.014340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee33a0f-5185-455d-9e26-e534137b1cd5 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:38.015695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18eb454-8f26-4eb4-aa17-9a44b9e9a604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:38.016554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18eb454-8f26-4eb4-aa17-9a44b9e9a604 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:38.017683 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18eb454-8f26-4eb4-aa17-9a44b9e9a604 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:38.017972 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18eb454-8f26-4eb4-aa17-9a44b9e9a604 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:38.030580 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18eb454-8f26-4eb4-aa17-9a44b9e9a604 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:38.035498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-e18eb454-8f26-4eb4-aa17-9a44b9e9a604 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:19:38.035498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:38.035498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:38.035498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:38.035498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:38.035498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:38.035498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:38.035498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:38.035498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:38.035498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:38.035498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:38.035498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:38.035498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:38.035498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:38.035498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:38.035498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:38.036663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:38.036663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:38.036663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:38.036663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:38.036663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:38.036663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:38.036663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:38.036663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:38.036663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:38.036663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:38.036663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:38.036663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:38.036663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:38.036663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:38.036663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:38.036663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:38.037522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:38.037522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:38.037522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:38.037522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:38.037522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:38.037522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:38.037522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:38.037522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:38.037522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:38.037522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:38.037522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 29 20:19:38.037522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:38.040160 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 186/358] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:38.040290 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffa574-04a7-4959-8f4b-2cbe0312d5fe None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:38.040506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffa574-04a7-4959-8f4b-2cbe0312d5fe None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:38.041297 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffa574-04a7-4959-8f4b-2cbe0312d5fe None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:38.043303 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 187/359] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:38.043303 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffa574-04a7-4959-8f4b-2cbe0312d5fe None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:38.061612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7bcf8-6027-4713-b1e0-744883c47f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:38.062297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7bcf8-6027-4713-b1e0-744883c47f42 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:38.062585 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7bcf8-6027-4713-b1e0-744883c47f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:38.062864 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7bcf8-6027-4713-b1e0-744883c47f42 None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:38.064173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7bcf8-6027-4713-b1e0-744883c47f42 None admin] RBAC: Authorizing `identity:get_project(project_id=eddee91f1cf44ccf83ca7cd9419ab306)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:38.065629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7bcf8-6027-4713-b1e0-744883c47f42 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:38.068425 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 173/360] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:37 2022] GET /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:38.069586 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e14c53-cda7-4074-8cd0-ba98923429fc None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:38.069904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e14c53-cda7-4074-8cd0-ba98923429fc None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:38.070139 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e14c53-cda7-4074-8cd0-ba98923429fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:38.093693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d7e14c53-cda7-4074-8cd0-ba98923429fc None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:38.094884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d7e14c53-cda7-4074-8cd0-ba98923429fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:38.122862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:38.175746 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffa574-04a7-4959-8f4b-2cbe0312d5fe None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:38.178798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:38.184520 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 174/361] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:38.186422 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e14c53-cda7-4074-8cd0-ba98923429fc None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:38.221751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140043c1-5288-4665-b8d2-68503f0915da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:38.222395 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140043c1-5288-4665-b8d2-68503f0915da None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:38.222666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140043c1-5288-4665-b8d2-68503f0915da None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:38.222953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140043c1-5288-4665-b8d2-68503f0915da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:38.223710 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140043c1-5288-4665-b8d2-68503f0915da None admin] RBAC: Adding query filter params (group.id=80832b8b4bff430eb00f2ad1f2236c03, role.id=578c9a17c0fa434f9602f5d2b3648785, scope.project.id=0370c388ce4e42d2ad10ebca467e24dc) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:38.224403 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140043c1-5288-4665-b8d2-68503f0915da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:38.225505 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140043c1-5288-4665-b8d2-68503f0915da None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:38.233662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 188/362] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Aug 29 20:19:38 2022] GET /identity/v3/role_assignments?group.id=80832b8b4bff430eb00f2ad1f2236c03&scope.project.id=0370c388ce4e42d2ad10ebca467e24dc&role.id=578c9a17c0fa434f9602f5d2b3648785 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:38.234897 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432b230-97fe-4cf4-8e57-9a8ba69c2d13 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:38.235188 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432b230-97fe-4cf4-8e57-9a8ba69c2d13 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:38.235465 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2432b230-97fe-4cf4-8e57-9a8ba69c2d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:38.254942 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2432b230-97fe-4cf4-8e57-9a8ba69c2d13 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:38.260505 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2432b230-97fe-4cf4-8e57-9a8ba69c2d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:38.347338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e14c53-cda7-4074-8cd0-ba98923429fc None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:38.378973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 189/363] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:38.380637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2432b230-97fe-4cf4-8e57-9a8ba69c2d13 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:38.403765 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87faea3-a9fd-4dcf-9cfa-5e2463e9cf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:38.405268 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87faea3-a9fd-4dcf-9cfa-5e2463e9cf49 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:38.405566 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87faea3-a9fd-4dcf-9cfa-5e2463e9cf49 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:38.405822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87faea3-a9fd-4dcf-9cfa-5e2463e9cf49 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:38.406487 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87faea3-a9fd-4dcf-9cfa-5e2463e9cf49 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:38.406788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87faea3-a9fd-4dcf-9cfa-5e2463e9cf49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:38.408195 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87faea3-a9fd-4dcf-9cfa-5e2463e9cf49 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:38.420097 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 175/364] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:38.426303 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87faea3-a9fd-4dcf-9cfa-5e2463e9cf49 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:38.458670 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:38.520593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:38.540784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2432b230-97fe-4cf4-8e57-9a8ba69c2d13 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:38.582872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87faea3-a9fd-4dcf-9cfa-5e2463e9cf49 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:38.590652 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b40c7c-eeaf-4fca-8a1d-b4e05295a726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:38.591304 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b40c7c-eeaf-4fca-8a1d-b4e05295a726 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:38.591588 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b40c7c-eeaf-4fca-8a1d-b4e05295a726 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:38.591866 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b40c7c-eeaf-4fca-8a1d-b4e05295a726 None admin] PATH_INFO: `/v3/groups/80832b8b4bff430eb00f2ad1f2236c03` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:38.596185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b40c7c-eeaf-4fca-8a1d-b4e05295a726 None admin] RBAC: Authorizing `identity:get_group(group_id=80832b8b4bff430eb00f2ad1f2236c03)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:38.597589 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b40c7c-eeaf-4fca-8a1d-b4e05295a726 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:38.604072 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 190/365] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 29 20:19:38 2022] GET /identity/v3/groups/80832b8b4bff430eb00f2ad1f2236c03 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:38.605302 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b40c7c-eeaf-4fca-8a1d-b4e05295a726 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:38.628772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2946226-c1a1-4000-982c-e70972669f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:38.629831 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2946226-c1a1-4000-982c-e70972669f0d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:38.630588 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2946226-c1a1-4000-982c-e70972669f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:38.631087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2946226-c1a1-4000-982c-e70972669f0d None admin] PATH_INFO: `/v3/users/f263250a762a40d4a6c2b8584298af35` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:38.632501 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2946226-c1a1-4000-982c-e70972669f0d None admin] RBAC: Authorizing `identity:get_user(user_id=f263250a762a40d4a6c2b8584298af35)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:38.634902 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2946226-c1a1-4000-982c-e70972669f0d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:38.638391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 176/366] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:19:38 2022] GET /identity/v3/users/f263250a762a40d4a6c2b8584298af35 => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:38.644364 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2946226-c1a1-4000-982c-e70972669f0d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:38.731766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b40c7c-eeaf-4fca-8a1d-b4e05295a726 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:38.756290 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:38.773202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e74e7-b892-4f0a-acfe-da20234a51a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:38.773925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e74e7-b892-4f0a-acfe-da20234a51a7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:38.774242 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e74e7-b892-4f0a-acfe-da20234a51a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:38.774523 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e74e7-b892-4f0a-acfe-da20234a51a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:38.775269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e74e7-b892-4f0a-acfe-da20234a51a7 None admin] RBAC: Adding query filter params (group.id=80832b8b4bff430eb00f2ad1f2236c03, role.id=89cf5ed93cc642588a54639342c59ea6, scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:38.775581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e74e7-b892-4f0a-acfe-da20234a51a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:38.777082 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e74e7-b892-4f0a-acfe-da20234a51a7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:38.788100 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 191/367] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Aug 29 20:19:38 2022] GET /identity/v3/role_assignments?group.id=80832b8b4bff430eb00f2ad1f2236c03&scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306&role.id=89cf5ed93cc642588a54639342c59ea6 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:38.789072 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e74e7-b892-4f0a-acfe-da20234a51a7 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:38.795898 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2946226-c1a1-4000-982c-e70972669f0d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:38.844808 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:38.855834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee02d77-7bb4-48e4-bcbf-03cca7096863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:38.856479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee02d77-7bb4-48e4-bcbf-03cca7096863 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:38.856766 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee02d77-7bb4-48e4-bcbf-03cca7096863 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:38.857075 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee02d77-7bb4-48e4-bcbf-03cca7096863 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:38.857818 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee02d77-7bb4-48e4-bcbf-03cca7096863 None admin] RBAC: Adding query filter params (role.id=5ae3132138594bb7a1fbacc9d93198c1, user.id=f263250a762a40d4a6c2b8584298af35, scope.system=all) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:38.858125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee02d77-7bb4-48e4-bcbf-03cca7096863 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:38.859597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee02d77-7bb4-48e4-bcbf-03cca7096863 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:38.881552 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 177/368] 10.222.0.18 () {56 vars in 1308 bytes} [Mon Aug 29 20:19:38 2022] GET /identity/v3/role_assignments?user.id=f263250a762a40d4a6c2b8584298af35&scope.system=all&role.id=5ae3132138594bb7a1fbacc9d93198c1 => generated 510 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:38.882922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee02d77-7bb4-48e4-bcbf-03cca7096863 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:38.946460 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e74e7-b892-4f0a-acfe-da20234a51a7 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:38.999579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d5650-a130-4a8c-bbd3-2ee7844e3855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:39.000364 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d5650-a130-4a8c-bbd3-2ee7844e3855 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:39.000558 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d5650-a130-4a8c-bbd3-2ee7844e3855 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:39.000829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d5650-a130-4a8c-bbd3-2ee7844e3855 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:39.017287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d5650-a130-4a8c-bbd3-2ee7844e3855 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:39.023041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-e58d5650-a130-4a8c-bbd3-2ee7844e3855 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:39.023041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:39.023041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:39.023041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:39.023041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:39.023041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:39.023041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:39.023041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:39.023041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:39.023041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:39.023041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:39.023041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:39.023041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:39.023041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:39.023041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:39.023041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:39.024079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:39.024079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:39.024079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:39.024079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:39.024079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:39.024079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:39.024079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:39.024079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:39.024079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:39.024079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:39.024079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:39.024079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:39.024079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:39.024079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:39.024079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:39.024079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:39.024911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:39.024911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:39.024911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:39.024911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:39.024911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:39.024911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:39.024911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:39.024911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:39.024911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:39.024911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:39.024911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:39.024911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:39.028463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 192/369] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:19:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:39.034430 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d5650-a130-4a8c-bbd3-2ee7844e3855 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:39.118706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:39.134054 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee02d77-7bb4-48e4-bcbf-03cca7096863 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:39.177684 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d5650-a130-4a8c-bbd3-2ee7844e3855 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:39.202518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fa527-2c31-437f-8b6f-6bfa2b5ead7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:39.203172 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fa527-2c31-437f-8b6f-6bfa2b5ead7a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:39.203453 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fa527-2c31-437f-8b6f-6bfa2b5ead7a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:39.203729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fa527-2c31-437f-8b6f-6bfa2b5ead7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:39.221152 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599fa527-2c31-437f-8b6f-6bfa2b5ead7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:39.227300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-599fa527-2c31-437f-8b6f-6bfa2b5ead7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:19:39.227300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:39.227300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:39.227300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:39.227300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:39.227300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:39.227300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:39.227300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:39.227300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:39.227300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:39.227300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:39.227300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:39.227300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:39.227300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:39.227300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:39.227300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:39.228240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:39.228240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:39.228240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:39.228240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:39.228240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:39.228240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:39.228240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:39.228240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:39.228240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:39.228240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:39.228240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:39.228240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:39.228240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:39.228240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:39.228240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:39.228240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:39.229018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:39.229018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:39.229018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:39.229018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:39.229018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:39.229018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:39.229018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:39.229018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:39.229018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:39.229018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:39.229018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:19:39.229018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:39.230885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 178/370] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:19:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:39.237933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07029e26-935a-4ccb-942a-917d1e208a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:39.238648 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07029e26-935a-4ccb-942a-917d1e208a43 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:39.238995 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07029e26-935a-4ccb-942a-917d1e208a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:39.239350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07029e26-935a-4ccb-942a-917d1e208a43 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:39.240141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07029e26-935a-4ccb-942a-917d1e208a43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:39.240510 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07029e26-935a-4ccb-942a-917d1e208a43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:39.241942 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07029e26-935a-4ccb-942a-917d1e208a43 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:39.246359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fa527-2c31-437f-8b6f-6bfa2b5ead7a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:39.256813 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 193/371] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:19:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:39.263096 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07029e26-935a-4ccb-942a-917d1e208a43 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:39.315750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:39.393492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fa527-2c31-437f-8b6f-6bfa2b5ead7a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:39.435293 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78132ecc-311c-4a50-985c-c1dfa7a96402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:39.435917 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78132ecc-311c-4a50-985c-c1dfa7a96402 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:39.436219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78132ecc-311c-4a50-985c-c1dfa7a96402 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:39.436486 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78132ecc-311c-4a50-985c-c1dfa7a96402 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:39.437157 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78132ecc-311c-4a50-985c-c1dfa7a96402 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:39.437454 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78132ecc-311c-4a50-985c-c1dfa7a96402 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:39.438829 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78132ecc-311c-4a50-985c-c1dfa7a96402 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:39.451103 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 179/372] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:39.456818 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78132ecc-311c-4a50-985c-c1dfa7a96402 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:39.490141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07029e26-935a-4ccb-942a-917d1e208a43 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:39.496429 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:39.569368 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78132ecc-311c-4a50-985c-c1dfa7a96402 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:39.574697 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffa696-f438-450a-9ee7-d84d28812cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:39.575341 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffa696-f438-450a-9ee7-d84d28812cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:39.575690 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffa696-f438-450a-9ee7-d84d28812cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:39.575996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffa696-f438-450a-9ee7-d84d28812cc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:39.580385 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:39.613286 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44588ec0-f600-42aa-a2aa-767284016a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:39.613887 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44588ec0-f600-42aa-a2aa-767284016a1c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:39.614163 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44588ec0-f600-42aa-a2aa-767284016a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:39.614430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44588ec0-f600-42aa-a2aa-767284016a1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:39.711293 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44588ec0-f600-42aa-a2aa-767284016a1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:39.715334 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44588ec0-f600-42aa-a2aa-767284016a1c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:39.721503 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffa696-f438-450a-9ee7-d84d28812cc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:39.722966 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffa696-f438-450a-9ee7-d84d28812cc9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:39.883177 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 180/373] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 20:19:39.891417 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44588ec0-f600-42aa-a2aa-767284016a1c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:39.912303 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:39.926286 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 194/374] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:39.931662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffa696-f438-450a-9ee7-d84d28812cc9 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:39.935665 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:40.086130 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44588ec0-f600-42aa-a2aa-767284016a1c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:40.101279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffa696-f438-450a-9ee7-d84d28812cc9 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:40.153927 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f339b3-c605-4834-914e-fb9e7b40756a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:40.154577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f339b3-c605-4834-914e-fb9e7b40756a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:40.154861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f339b3-c605-4834-914e-fb9e7b40756a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:40.155659 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b659f-0465-48b5-8856-173f797d4d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:40.156997 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b659f-0465-48b5-8856-173f797d4d95 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:40.157101 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f339b3-c605-4834-914e-fb9e7b40756a None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:40.157175 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b659f-0465-48b5-8856-173f797d4d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:40.157175 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b659f-0465-48b5-8856-173f797d4d95 None admin] PATH_INFO: `/v3/projects/31796c851dfa4e61a8e1afef74d31165` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:40.158163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f339b3-c605-4834-914e-fb9e7b40756a None admin] RBAC: Authorizing `identity:get_project(project_id=eddee91f1cf44ccf83ca7cd9419ab306)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:40.159819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f339b3-c605-4834-914e-fb9e7b40756a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:40.160566 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b659f-0465-48b5-8856-173f797d4d95 None admin] RBAC: Authorizing `identity:get_project(project_id=31796c851dfa4e61a8e1afef74d31165)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:40.162314 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b659f-0465-48b5-8856-173f797d4d95 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:40.164230 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 181/376] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:39 2022] GET /identity/v3/projects/31796c851dfa4e61a8e1afef74d31165 => generated 344 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:40.164406 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 195/376] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:39 2022] GET /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:40.167557 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b659f-0465-48b5-8856-173f797d4d95 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:40.168338 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fed9f2-1b4d-4978-a6fa-2ac90bcd0dd6 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:40.168957 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fed9f2-1b4d-4978-a6fa-2ac90bcd0dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:40.169247 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fed9f2-1b4d-4978-a6fa-2ac90bcd0dd6 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:40.171532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 196/377] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:40.171532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fed9f2-1b4d-4978-a6fa-2ac90bcd0dd6 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:40.263225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:40.273000 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:40.306045 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b659f-0465-48b5-8856-173f797d4d95 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:40.316326 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fed9f2-1b4d-4978-a6fa-2ac90bcd0dd6 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:40.356259 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41814610-aec3-44a3-84d9-ec4bf1317f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:40.356879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41814610-aec3-44a3-84d9-ec4bf1317f5c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:40.357161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41814610-aec3-44a3-84d9-ec4bf1317f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:40.357436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41814610-aec3-44a3-84d9-ec4bf1317f5c None admin] PATH_INFO: `/v3/groups/80832b8b4bff430eb00f2ad1f2236c03` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:40.359214 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41814610-aec3-44a3-84d9-ec4bf1317f5c None admin] RBAC: Authorizing `identity:get_group(group_id=80832b8b4bff430eb00f2ad1f2236c03)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:40.360530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41814610-aec3-44a3-84d9-ec4bf1317f5c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:40.361201 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d3a5f-30f7-4daf-9043-9380011056e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:40.362103 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d3a5f-30f7-4daf-9043-9380011056e8 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:40.362469 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 182/378] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 29 20:19:40 2022] GET /identity/v3/groups/80832b8b4bff430eb00f2ad1f2236c03 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:40.362841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d3a5f-30f7-4daf-9043-9380011056e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:40.363390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d3a5f-30f7-4daf-9043-9380011056e8 None admin] PATH_INFO: `/v3/groups/8c2f2e2765a54038bbe175747ee30305` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:40.363728 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47f019-ab4a-4454-9f4c-475cce311d7b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:40.364042 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47f019-ab4a-4454-9f4c-475cce311d7b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:40.364317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47f019-ab4a-4454-9f4c-475cce311d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:40.371138 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d3a5f-30f7-4daf-9043-9380011056e8 None admin] RBAC: Authorizing `identity:get_group(group_id=8c2f2e2765a54038bbe175747ee30305)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:40.372928 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d3a5f-30f7-4daf-9043-9380011056e8 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:40.375058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 197/379] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 29 20:19:40 2022] GET /identity/v3/groups/8c2f2e2765a54038bbe175747ee30305 => generated 229 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:40.376082 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d3a5f-30f7-4daf-9043-9380011056e8 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:40.388609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-3d47f019-ab4a-4454-9f4c-475cce311d7b None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:40.389906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-3d47f019-ab4a-4454-9f4c-475cce311d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:40.483349 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 183/380] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:40.484661 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47f019-ab4a-4454-9f4c-475cce311d7b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:40.509017 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d3a5f-30f7-4daf-9043-9380011056e8 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:40.577741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c7c37-8509-43dc-888e-47e61995200d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:40.578752 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c7c37-8509-43dc-888e-47e61995200d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:40.579237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c7c37-8509-43dc-888e-47e61995200d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:40.580280 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c7c37-8509-43dc-888e-47e61995200d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:40.581227 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c7c37-8509-43dc-888e-47e61995200d None admin] RBAC: Adding query filter params (group.id=80832b8b4bff430eb00f2ad1f2236c03, role.id=578c9a17c0fa434f9602f5d2b3648785, scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:40.581863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c7c37-8509-43dc-888e-47e61995200d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:40.583676 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c7c37-8509-43dc-888e-47e61995200d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:40.589226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:40.597675 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 198/381] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Aug 29 20:19:40 2022] GET /identity/v3/role_assignments?group.id=80832b8b4bff430eb00f2ad1f2236c03&scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306&role.id=578c9a17c0fa434f9602f5d2b3648785 => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:40.618498 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:40.643083 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47f019-ab4a-4454-9f4c-475cce311d7b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:40.685165 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69020a88-bcb4-43d5-b565-9b0c7e2b6129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:40.685780 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69020a88-bcb4-43d5-b565-9b0c7e2b6129 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:40.686056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69020a88-bcb4-43d5-b565-9b0c7e2b6129 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:40.686325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69020a88-bcb4-43d5-b565-9b0c7e2b6129 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:40.687053 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69020a88-bcb4-43d5-b565-9b0c7e2b6129 None admin] RBAC: Adding query filter params (group.id=8c2f2e2765a54038bbe175747ee30305, role.id=322da7bd64024770a64bdb73e5c54719, scope.project.id=31796c851dfa4e61a8e1afef74d31165) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:40.687352 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69020a88-bcb4-43d5-b565-9b0c7e2b6129 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:40.688807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69020a88-bcb4-43d5-b565-9b0c7e2b6129 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:40.701736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 184/382] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Aug 29 20:19:40 2022] GET /identity/v3/role_assignments?group.id=8c2f2e2765a54038bbe175747ee30305&scope.project.id=31796c851dfa4e61a8e1afef74d31165&role.id=322da7bd64024770a64bdb73e5c54719 => generated 260 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:40.852594 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c7c37-8509-43dc-888e-47e61995200d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:40.988518 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c7c37-8509-43dc-888e-47e61995200d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:41.029350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3f689-4b1f-4fab-9f08-b6640db68d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:41.030063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3f689-4b1f-4fab-9f08-b6640db68d43 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:41.030346 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3f689-4b1f-4fab-9f08-b6640db68d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:41.030623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3f689-4b1f-4fab-9f08-b6640db68d43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:41.044086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3f689-4b1f-4fab-9f08-b6640db68d43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:41.049482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-0ce3f689-4b1f-4fab-9f08-b6640db68d43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:41.049482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:41.049482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:41.049482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:41.049482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:41.049482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:41.049482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:41.049482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:41.049482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:41.049482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:41.049482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:41.049482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:41.049482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:41.049482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:41.049482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:41.049482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:41.050372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:41.050372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:41.050372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:41.050372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:41.050372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:41.050372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:41.050372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:41.050372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:41.050372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:41.050372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:41.050372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:41.050372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:41.050372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:41.050372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:41.050372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:41.050372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:41.051554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:41.051554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:41.051554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:41.051554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:41.051554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:41.051554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:41.051554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:41.051554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:41.051554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:41.051554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:41.051554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:41.051554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:41.051554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 199/383] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:41.058921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69020a88-bcb4-43d5-b565-9b0c7e2b6129 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:41.096199 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6ea06-f25d-4b9d-84f3-15f80024b7cc None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:41.096533 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6ea06-f25d-4b9d-84f3-15f80024b7cc None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:41.096759 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6ea06-f25d-4b9d-84f3-15f80024b7cc None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:41.098175 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 200/384] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:19:41.103576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1442c-c1fa-4681-97bd-f3ccec07854d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:41.104186 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1442c-c1fa-4681-97bd-f3ccec07854d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:41.104186 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1442c-c1fa-4681-97bd-f3ccec07854d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:41.132256 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-44a1442c-c1fa-4681-97bd-f3ccec07854d None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:41.133682 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-44a1442c-c1fa-4681-97bd-f3ccec07854d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:41.242802 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:41.245987 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69020a88-bcb4-43d5-b565-9b0c7e2b6129 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:41.267913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 201/385] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 29 20:19:41.305321 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c63a4-d9fb-4de1-b800-091e61b40f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:41.306172 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c63a4-d9fb-4de1-b800-091e61b40f91 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:41.306622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c63a4-d9fb-4de1-b800-091e61b40f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:41.307073 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c63a4-d9fb-4de1-b800-091e61b40f91 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:41.308117 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c63a4-d9fb-4de1-b800-091e61b40f91 None admin] RBAC: Adding query filter params (name=member) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:41.308587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c63a4-d9fb-4de1-b800-091e61b40f91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:41.310195 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c63a4-d9fb-4de1-b800-091e61b40f91 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:41.324876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 185/386] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:41.330540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1442c-c1fa-4681-97bd-f3ccec07854d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:41.461177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1442c-c1fa-4681-97bd-f3ccec07854d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:41.510137 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b6be7-8c25-4cef-bcfb-b7b4c4ed7d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:41.511098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b6be7-8c25-4cef-bcfb-b7b4c4ed7d30 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:41.511554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b6be7-8c25-4cef-bcfb-b7b4c4ed7d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:41.512027 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b6be7-8c25-4cef-bcfb-b7b4c4ed7d30 None admin] PATH_INFO: `/v3/groups/80832b8b4bff430eb00f2ad1f2236c03` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:41.514491 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b6be7-8c25-4cef-bcfb-b7b4c4ed7d30 None admin] RBAC: Authorizing `identity:get_group(group_id=80832b8b4bff430eb00f2ad1f2236c03)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:41.516233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b6be7-8c25-4cef-bcfb-b7b4c4ed7d30 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:41.518359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 202/387] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 29 20:19:41 2022] GET /identity/v3/groups/80832b8b4bff430eb00f2ad1f2236c03 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:41.523799 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c63a4-d9fb-4de1-b800-091e61b40f91 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:41.527012 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:41.643186 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c63a4-d9fb-4de1-b800-091e61b40f91 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:41.653156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0ce08-bcac-43f8-8689-f162948f2e37 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:41.653725 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0ce08-bcac-43f8-8689-f162948f2e37 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:41.654177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0ce08-bcac-43f8-8689-f162948f2e37 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:41.655745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 203/388] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:41.662464 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c41fd-be4d-4701-a772-961e31310780 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:41.662948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c41fd-be4d-4701-a772-961e31310780 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:41.663398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c41fd-be4d-4701-a772-961e31310780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:41.687925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-6a4c41fd-be4d-4701-a772-961e31310780 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:41.693007 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6a4c41fd-be4d-4701-a772-961e31310780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:41.708460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c8915-b03d-4c92-9bc8-0afb3e29be70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:41.709126 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c8915-b03d-4c92-9bc8-0afb3e29be70 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:41.709401 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c8915-b03d-4c92-9bc8-0afb3e29be70 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:41.709672 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c8915-b03d-4c92-9bc8-0afb3e29be70 None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:41.710785 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c8915-b03d-4c92-9bc8-0afb3e29be70 None admin] RBAC: Authorizing `identity:get_project(project_id=0370c388ce4e42d2ad10ebca467e24dc)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:41.712260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c8915-b03d-4c92-9bc8-0afb3e29be70 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:41.714693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 186/389] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:41 2022] GET /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:41.715312 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c8915-b03d-4c92-9bc8-0afb3e29be70 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:41.757102 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:41.790232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 204/390] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:19:41.791533 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c41fd-be4d-4701-a772-961e31310780 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:41.820336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:41.872301 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c8915-b03d-4c92-9bc8-0afb3e29be70 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:41.919042 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6b6e0-aba8-4733-9eb4-f9795b97a214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:41.919704 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6b6e0-aba8-4733-9eb4-f9795b97a214 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:41.920014 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6b6e0-aba8-4733-9eb4-f9795b97a214 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:41.920295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6b6e0-aba8-4733-9eb4-f9795b97a214 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:41.924538 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c41fd-be4d-4701-a772-961e31310780 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:41.944355 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6b6e0-aba8-4733-9eb4-f9795b97a214 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:41.951968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-53f6b6e0-aba8-4733-9eb4-f9795b97a214 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:41.951968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:41.951968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:41.951968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:41.951968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:41.951968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:41.951968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:41.951968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:41.951968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:41.951968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:41.951968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:41.951968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:41.951968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:41.951968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:41.951968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:41.951968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:41.953182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:41.953182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:41.953182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:41.953182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:41.953182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:41.953182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:41.953182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:41.953182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:41.953182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:41.953182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:41.953182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:41.953182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:41.953182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:41.953182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:41.953182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:41.953182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:41.954164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:41.954164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:41.954164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:41.954164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:41.954164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:41.954164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:41.954164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:41.954164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:41.954164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:41.954164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:41.954164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:41.954164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:41.954164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 187/391] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:19:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:41.967559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6b6e0-aba8-4733-9eb4-f9795b97a214 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:41.971236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe477d-b3dd-42f5-a468-af38ddfe4903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:41.972213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe477d-b3dd-42f5-a468-af38ddfe4903 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:41.972638 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe477d-b3dd-42f5-a468-af38ddfe4903 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:41.973049 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe477d-b3dd-42f5-a468-af38ddfe4903 None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc/groups/80832b8b4bff430eb00f2ad1f2236c03/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:41.979900 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe477d-b3dd-42f5-a468-af38ddfe4903 None admin] RBAC: Authorizing `identity:create_grant(project_id=0370c388ce4e42d2ad10ebca467e24dc, group_id=80832b8b4bff430eb00f2ad1f2236c03, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:41.981556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe477d-b3dd-42f5-a468-af38ddfe4903 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:41.999535 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 205/392] 10.222.0.18 () {58 vars in 1442 bytes} [Mon Aug 29 20:19:41 2022] PUT /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc/groups/80832b8b4bff430eb00f2ad1f2236c03/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:19:42.126685 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6b6e0-aba8-4733-9eb4-f9795b97a214 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:42.165915 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:42.176757 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6829f-89f9-46d0-903f-63fb73ac2ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:42.177375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6829f-89f9-46d0-903f-63fb73ac2ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:42.177693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6829f-89f9-46d0-903f-63fb73ac2ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:42.177928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6829f-89f9-46d0-903f-63fb73ac2ff7 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:42.178692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6829f-89f9-46d0-903f-63fb73ac2ff7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:42.179005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6829f-89f9-46d0-903f-63fb73ac2ff7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:42.180417 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6829f-89f9-46d0-903f-63fb73ac2ff7 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:42.189875 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 188/393] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:19:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:42.197227 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe477d-b3dd-42f5-a468-af38ddfe4903 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:42.204711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6829f-89f9-46d0-903f-63fb73ac2ff7 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:42.297119 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:42.338323 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6829f-89f9-46d0-903f-63fb73ac2ff7 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:42.368564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe477d-b3dd-42f5-a468-af38ddfe4903 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:42.401068 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bb34e-a6f6-413c-90ff-463f81ca3f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:42.402090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bb34e-a6f6-413c-90ff-463f81ca3f5c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:42.402752 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bb34e-a6f6-413c-90ff-463f81ca3f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:42.403906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bb34e-a6f6-413c-90ff-463f81ca3f5c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:42.423121 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bb34e-a6f6-413c-90ff-463f81ca3f5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:42.423574 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e67fd-f92a-4830-ba66-63afffc39f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:42.424374 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e67fd-f92a-4830-ba66-63afffc39f29 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:42.424753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e67fd-f92a-4830-ba66-63afffc39f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:42.425128 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e67fd-f92a-4830-ba66-63afffc39f29 None admin] PATH_INFO: `/v3/groups/80832b8b4bff430eb00f2ad1f2236c03` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:42.426342 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e67fd-f92a-4830-ba66-63afffc39f29 None admin] RBAC: Authorizing `identity:get_group(group_id=80832b8b4bff430eb00f2ad1f2236c03)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:42.427879 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8e67fd-f92a-4830-ba66-63afffc39f29 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:42.429585 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 206/394] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 29 20:19:42 2022] GET /identity/v3/groups/80832b8b4bff430eb00f2ad1f2236c03 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:42.430091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-3d0bb34e-a6f6-413c-90ff-463f81ca3f5c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:42.430091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:42.430091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:42.430091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:42.430091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:42.430091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:42.430091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:42.430091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:42.430091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:42.430091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:42.430091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:42.430091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:42.430091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:42.430091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:42.430091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:42.430091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:42.430939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:42.430939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:42.430939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:42.430939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:42.430939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:42.430939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:42.430939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:42.430939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:42.430939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:42.430939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:42.430939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:42.430939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:42.430939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:42.430939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:42.430939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:42.430939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:42.431723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:42.431723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:42.431723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:42.431723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:42.431723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:42.431723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:42.431723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:42.431723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:42.431723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:42.431723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:42.431723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:42.431723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:42.434399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 189/395] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:42.440265 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e67fd-f92a-4830-ba66-63afffc39f29 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:42.440430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bb34e-a6f6-413c-90ff-463f81ca3f5c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:42.506999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:42.565300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bb34e-a6f6-413c-90ff-463f81ca3f5c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:42.606211 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa589a-5209-4b5e-b0b9-ba803aff8272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:42.607389 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa589a-5209-4b5e-b0b9-ba803aff8272 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:42.608248 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa589a-5209-4b5e-b0b9-ba803aff8272 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:42.608538 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa589a-5209-4b5e-b0b9-ba803aff8272 None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:42.609688 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa589a-5209-4b5e-b0b9-ba803aff8272 None admin] RBAC: Authorizing `identity:get_project(project_id=0370c388ce4e42d2ad10ebca467e24dc)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:42.611252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa589a-5209-4b5e-b0b9-ba803aff8272 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:42.613278 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 190/396] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:42 2022] GET /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:42.620962 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa589a-5209-4b5e-b0b9-ba803aff8272 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:42.647407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8e67fd-f92a-4830-ba66-63afffc39f29 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:42.683559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:42.724161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806af12-172d-4417-a63c-1a447c72662d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:42.724838 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806af12-172d-4417-a63c-1a447c72662d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:42.725127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806af12-172d-4417-a63c-1a447c72662d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:42.725404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806af12-172d-4417-a63c-1a447c72662d None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:42.726082 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806af12-172d-4417-a63c-1a447c72662d None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:42.726388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806af12-172d-4417-a63c-1a447c72662d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:42.727828 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806af12-172d-4417-a63c-1a447c72662d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:42.750485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 207/397] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:42.755248 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806af12-172d-4417-a63c-1a447c72662d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:42.817691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa589a-5209-4b5e-b0b9-ba803aff8272 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:42.860529 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:42.890157 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551b9d8-2e26-408c-986c-e7a8714636ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:42.890831 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551b9d8-2e26-408c-986c-e7a8714636ba None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:42.891113 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551b9d8-2e26-408c-986c-e7a8714636ba None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:42.891390 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551b9d8-2e26-408c-986c-e7a8714636ba None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc/groups/80832b8b4bff430eb00f2ad1f2236c03/roles/578c9a17c0fa434f9602f5d2b3648785` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:42.896069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551b9d8-2e26-408c-986c-e7a8714636ba None admin] RBAC: Authorizing `identity:create_grant(project_id=0370c388ce4e42d2ad10ebca467e24dc, group_id=80832b8b4bff430eb00f2ad1f2236c03, role_id=578c9a17c0fa434f9602f5d2b3648785)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:42.897538 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551b9d8-2e26-408c-986c-e7a8714636ba None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:42.923177 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 191/398] 10.222.0.18 () {58 vars in 1442 bytes} [Mon Aug 29 20:19:42 2022] PUT /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc/groups/80832b8b4bff430eb00f2ad1f2236c03/roles/578c9a17c0fa434f9602f5d2b3648785 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:19:42.952049 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806af12-172d-4417-a63c-1a447c72662d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:43.004350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f48ef-16d8-43af-9a64-0e26ddbc5b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:43.005018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f48ef-16d8-43af-9a64-0e26ddbc5b65 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:43.005305 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f48ef-16d8-43af-9a64-0e26ddbc5b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:43.005584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f48ef-16d8-43af-9a64-0e26ddbc5b65 None admin] PATH_INFO: `/v3/groups/80832b8b4bff430eb00f2ad1f2236c03` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:43.012900 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f48ef-16d8-43af-9a64-0e26ddbc5b65 None admin] RBAC: Authorizing `identity:get_group(group_id=80832b8b4bff430eb00f2ad1f2236c03)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:43.014389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f48ef-16d8-43af-9a64-0e26ddbc5b65 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:43.016489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 208/399] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 29 20:19:42 2022] GET /identity/v3/groups/80832b8b4bff430eb00f2ad1f2236c03 => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:43.021968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551b9d8-2e26-408c-986c-e7a8714636ba None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:43.065807 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7d08d-ed3d-460a-977b-64943af50cb5 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:43.066114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7d08d-ed3d-460a-977b-64943af50cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:43.066372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7d08d-ed3d-460a-977b-64943af50cb5 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:43.068134 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 209/400] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:43.073706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a6eda-9938-4945-a994-124cfbebd07e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:43.074002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a6eda-9938-4945-a994-124cfbebd07e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:43.074284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a6eda-9938-4945-a994-124cfbebd07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:43.113056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b19a6eda-9938-4945-a994-124cfbebd07e None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:43.114290 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b19a6eda-9938-4945-a994-124cfbebd07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:43.176113 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551b9d8-2e26-408c-986c-e7a8714636ba None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:43.191377 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:43.225444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e784b9-abc0-42a3-bee5-839b68147454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:43.226249 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e784b9-abc0-42a3-bee5-839b68147454 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:43.226529 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e784b9-abc0-42a3-bee5-839b68147454 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:43.226796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e784b9-abc0-42a3-bee5-839b68147454 None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:43.228850 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e784b9-abc0-42a3-bee5-839b68147454 None admin] RBAC: Authorizing `identity:get_project(project_id=eddee91f1cf44ccf83ca7cd9419ab306)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:43.231070 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e784b9-abc0-42a3-bee5-839b68147454 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:43.231348 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 210/401] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:19:43.233841 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 192/402] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:43 2022] GET /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:43.241545 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a6eda-9938-4945-a994-124cfbebd07e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:43.395651 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a6eda-9938-4945-a994-124cfbebd07e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:43.440781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d0b25-bf8c-4c45-9bed-4383928ad0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:43.441453 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d0b25-bf8c-4c45-9bed-4383928ad0e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:43.441739 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d0b25-bf8c-4c45-9bed-4383928ad0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:43.442014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1d0b25-bf8c-4c45-9bed-4383928ad0e6 None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306/groups/80832b8b4bff430eb00f2ad1f2236c03/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:43.446922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17985aa-9f87-4408-a831-68a86f3749cd None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:43.447376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17985aa-9f87-4408-a831-68a86f3749cd None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:43.447452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17985aa-9f87-4408-a831-68a86f3749cd None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:43.449100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 193/403] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:43.449421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d0b25-bf8c-4c45-9bed-4383928ad0e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=eddee91f1cf44ccf83ca7cd9419ab306, group_id=80832b8b4bff430eb00f2ad1f2236c03, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:43.450873 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1d0b25-bf8c-4c45-9bed-4383928ad0e6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:43.454835 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a478de-4df7-43c9-aa53-d238e799f0c3 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:43.455121 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a478de-4df7-43c9-aa53-d238e799f0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:43.455394 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a478de-4df7-43c9-aa53-d238e799f0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:43.467167 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 211/404] 10.222.0.18 () {58 vars in 1442 bytes} [Mon Aug 29 20:19:43 2022] PUT /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306/groups/80832b8b4bff430eb00f2ad1f2236c03/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:19:43.480495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-30a478de-4df7-43c9-aa53-d238e799f0c3 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:43.481623 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-30a478de-4df7-43c9-aa53-d238e799f0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:43.539255 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:43.569386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 194/405] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:19:43.705864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d0b25-bf8c-4c45-9bed-4383928ad0e6 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:43.744390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:43.821473 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1d0b25-bf8c-4c45-9bed-4383928ad0e6 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:43.863445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448c5a2-5aa7-44de-8b20-045b7582d87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:43.864117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448c5a2-5aa7-44de-8b20-045b7582d87a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:43.864409 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448c5a2-5aa7-44de-8b20-045b7582d87a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:43.864691 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448c5a2-5aa7-44de-8b20-045b7582d87a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:43.880296 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448c5a2-5aa7-44de-8b20-045b7582d87a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:43.885362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-4448c5a2-5aa7-44de-8b20-045b7582d87a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:43.885362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:43.885362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:43.885362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:43.885362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:43.885362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:43.885362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:43.885362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:43.885362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:43.885362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:43.885362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:43.885362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:43.885362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:43.885362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:43.885362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:43.885362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:43.886783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:43.886783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:43.886783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:43.886783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:43.886783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:43.886783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:43.886783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:43.886783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:43.886783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:43.886783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:43.886783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:43.886783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:43.886783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:43.886783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:43.886783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:43.886783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:43.887623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:43.887623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:43.887623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:43.887623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:43.887623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:43.887623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:43.887623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:43.887623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:43.887623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:43.887623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:43.887623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:43.887623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:43.887623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 212/406] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:19:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:43.892898 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a478de-4df7-43c9-aa53-d238e799f0c3 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:43.974537 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448c5a2-5aa7-44de-8b20-045b7582d87a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:44.019024 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a478de-4df7-43c9-aa53-d238e799f0c3 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:44.057063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01792c-4bd5-42a1-9fca-1b977d64bcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:44.059401 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01792c-4bd5-42a1-9fca-1b977d64bcd1 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:44.059891 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01792c-4bd5-42a1-9fca-1b977d64bcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:44.060381 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01792c-4bd5-42a1-9fca-1b977d64bcd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:44.061413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01792c-4bd5-42a1-9fca-1b977d64bcd1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:44.063852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01792c-4bd5-42a1-9fca-1b977d64bcd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:44.065455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01792c-4bd5-42a1-9fca-1b977d64bcd1 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:44.082641 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 195/407] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:19:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:44.089480 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01792c-4bd5-42a1-9fca-1b977d64bcd1 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:44.143246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:44.163120 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:44.173352 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448c5a2-5aa7-44de-8b20-045b7582d87a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:44.208066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01792c-4bd5-42a1-9fca-1b977d64bcd1 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:44.243609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d86d8-612d-4c13-9451-3a5f5461d96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:44.243975 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad085771-2d1b-4155-b8b2-78894223a129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:44.244678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad085771-2d1b-4155-b8b2-78894223a129 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:44.244967 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad085771-2d1b-4155-b8b2-78894223a129 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:44.245775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad085771-2d1b-4155-b8b2-78894223a129 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:44.246300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d86d8-612d-4c13-9451-3a5f5461d96f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:44.246769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d86d8-612d-4c13-9451-3a5f5461d96f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:44.247228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d86d8-612d-4c13-9451-3a5f5461d96f None admin] PATH_INFO: `/v3/groups/80832b8b4bff430eb00f2ad1f2236c03` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:44.248603 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518d86d8-612d-4c13-9451-3a5f5461d96f None admin] RBAC: Authorizing `identity:get_group(group_id=80832b8b4bff430eb00f2ad1f2236c03)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:44.250243 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518d86d8-612d-4c13-9451-3a5f5461d96f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:44.252056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 196/408] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 29 20:19:44 2022] GET /identity/v3/groups/80832b8b4bff430eb00f2ad1f2236c03 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:44.259167 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d86d8-612d-4c13-9451-3a5f5461d96f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:44.273637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad085771-2d1b-4155-b8b2-78894223a129 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:44.279208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-ad085771-2d1b-4155-b8b2-78894223a129 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:19:44.279208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:44.279208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:44.279208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:44.279208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:44.279208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:44.279208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:44.279208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:44.279208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:44.279208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:44.279208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:44.279208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:44.279208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:44.279208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:44.279208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:44.279208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:44.280099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:44.280099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:44.280099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:44.280099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:44.280099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:44.280099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:44.280099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:44.280099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:44.280099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:44.280099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:44.280099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:44.280099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:44.280099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:44.280099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:44.280099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:44.280099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:44.280950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:44.280950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:44.280950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:44.280950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:44.280950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:44.280950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:44.280950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:44.280950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:44.280950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:44.280950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:44.280950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:19:44.280950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:44.294208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 213/409] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:19:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:19:44.302922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad085771-2d1b-4155-b8b2-78894223a129 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:44.393759 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d86d8-612d-4c13-9451-3a5f5461d96f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:44.403952 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:44.432526 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b571bf-f558-45da-999d-02149d023199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:44.433478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b571bf-f558-45da-999d-02149d023199 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:44.433855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b571bf-f558-45da-999d-02149d023199 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:44.434223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b571bf-f558-45da-999d-02149d023199 None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:44.435506 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b571bf-f558-45da-999d-02149d023199 None admin] RBAC: Authorizing `identity:get_project(project_id=eddee91f1cf44ccf83ca7cd9419ab306)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:44.437221 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b571bf-f558-45da-999d-02149d023199 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:44.439405 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 197/410] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:44 2022] GET /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:44.448076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b571bf-f558-45da-999d-02149d023199 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:44.521380 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad085771-2d1b-4155-b8b2-78894223a129 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:44.557687 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b571bf-f558-45da-999d-02149d023199 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:44.591195 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4fa23-749c-4695-bd75-19413ee34772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:44.592192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4fa23-749c-4695-bd75-19413ee34772 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:44.592666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4fa23-749c-4695-bd75-19413ee34772 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:44.593110 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4fa23-749c-4695-bd75-19413ee34772 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:44.593970 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4fa23-749c-4695-bd75-19413ee34772 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:44.594456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd153002-bea1-4a89-9848-e052dd0cef27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:44.594573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4fa23-749c-4695-bd75-19413ee34772 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:44.595217 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd153002-bea1-4a89-9848-e052dd0cef27 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:44.595576 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd153002-bea1-4a89-9848-e052dd0cef27 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:44.595993 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd153002-bea1-4a89-9848-e052dd0cef27 None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306/groups/80832b8b4bff430eb00f2ad1f2236c03/roles/578c9a17c0fa434f9602f5d2b3648785` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:44.596695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4fa23-749c-4695-bd75-19413ee34772 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:44.602333 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd153002-bea1-4a89-9848-e052dd0cef27 None admin] RBAC: Authorizing `identity:create_grant(project_id=eddee91f1cf44ccf83ca7cd9419ab306, group_id=80832b8b4bff430eb00f2ad1f2236c03, role_id=578c9a17c0fa434f9602f5d2b3648785)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:44.602333 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd153002-bea1-4a89-9848-e052dd0cef27 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:44.608790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 214/411] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:44.610353 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408eaf8-8783-4076-b9ae-c82b358922d4 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:44.610652 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408eaf8-8783-4076-b9ae-c82b358922d4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:44.611041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408eaf8-8783-4076-b9ae-c82b358922d4 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:44.612384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 215/412] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:44.616077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 198/413] 10.222.0.18 () {58 vars in 1442 bytes} [Mon Aug 29 20:19:44 2022] PUT /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306/groups/80832b8b4bff430eb00f2ad1f2236c03/roles/578c9a17c0fa434f9602f5d2b3648785 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:19:44.616929 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408eaf8-8783-4076-b9ae-c82b358922d4 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:44.619266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73493846-eccc-4bd4-bccd-e8891e7bf664 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:44.619823 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73493846-eccc-4bd4-bccd-e8891e7bf664 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:44.620329 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73493846-eccc-4bd4-bccd-e8891e7bf664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:44.643144 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-73493846-eccc-4bd4-bccd-e8891e7bf664 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:44.644794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-73493846-eccc-4bd4-bccd-e8891e7bf664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:44.667537 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:44.683203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:44.729802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 199/414] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:19:44.827150 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408eaf8-8783-4076-b9ae-c82b358922d4 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:44.868582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62bc7f8-619f-433a-ae00-ef6db5667c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:44.869285 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bc7f8-619f-433a-ae00-ef6db5667c8c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:44.869566 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bc7f8-619f-433a-ae00-ef6db5667c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:44.869841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bc7f8-619f-433a-ae00-ef6db5667c8c None admin] PATH_INFO: `/v3/groups/8c2f2e2765a54038bbe175747ee30305` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:44.871494 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62bc7f8-619f-433a-ae00-ef6db5667c8c None admin] RBAC: Authorizing `identity:get_group(group_id=8c2f2e2765a54038bbe175747ee30305)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:44.872965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62bc7f8-619f-433a-ae00-ef6db5667c8c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:44.874854 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 216/415] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 29 20:19:44 2022] GET /identity/v3/groups/8c2f2e2765a54038bbe175747ee30305 => generated 229 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:44.880250 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73493846-eccc-4bd4-bccd-e8891e7bf664 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:45.017452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73493846-eccc-4bd4-bccd-e8891e7bf664 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:45.028269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62bc7f8-619f-433a-ae00-ef6db5667c8c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:45.074608 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f07be-bd27-412c-9acc-fcd8e981113d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:45.075263 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f07be-bd27-412c-9acc-fcd8e981113d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:45.075542 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f07be-bd27-412c-9acc-fcd8e981113d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:45.075815 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f07be-bd27-412c-9acc-fcd8e981113d None admin] PATH_INFO: `/v3/projects/31796c851dfa4e61a8e1afef74d31165` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:45.077104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f07be-bd27-412c-9acc-fcd8e981113d None admin] RBAC: Authorizing `identity:get_project(project_id=31796c851dfa4e61a8e1afef74d31165)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:45.078623 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f07be-bd27-412c-9acc-fcd8e981113d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:45.082543 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 200/416] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:44 2022] GET /identity/v3/projects/31796c851dfa4e61a8e1afef74d31165 => generated 344 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:45.093040 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f07be-bd27-412c-9acc-fcd8e981113d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:45.154604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:45.189094 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:45.215817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62bc7f8-619f-433a-ae00-ef6db5667c8c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:45.258555 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f07be-bd27-412c-9acc-fcd8e981113d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:45.269360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c61513-ae5f-46f8-b6b4-7945f0628c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:45.270059 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c61513-ae5f-46f8-b6b4-7945f0628c72 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:45.270344 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c61513-ae5f-46f8-b6b4-7945f0628c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:45.270671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c61513-ae5f-46f8-b6b4-7945f0628c72 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:45.290989 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c61513-ae5f-46f8-b6b4-7945f0628c72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:45.299295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-64c61513-ae5f-46f8-b6b4-7945f0628c72 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:45.299295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:45.299295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:45.299295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:45.299295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:45.299295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:45.299295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:45.299295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:45.299295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:45.299295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:45.299295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:45.299295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:45.299295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:45.299295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:45.299295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:45.299295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:45.301233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:45.301233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:45.301233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:45.301233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:45.301233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:45.301233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:45.301233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:45.301233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:45.301233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:45.301233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:45.301233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:45.301233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:45.301233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:45.301233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:45.301233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:45.301233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:45.302031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:45.302031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:45.302031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:45.302031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:45.302031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:45.302031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:45.302031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:45.302031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:45.302031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:45.302031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:45.302031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:45.302031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:45.302031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 217/417] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:45.312173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf7f2d-4f2c-4836-aeb9-8e6dfc04d4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:45.313604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c61513-ae5f-46f8-b6b4-7945f0628c72 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:45.314982 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf7f2d-4f2c-4836-aeb9-8e6dfc04d4ca None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:45.317284 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf7f2d-4f2c-4836-aeb9-8e6dfc04d4ca None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:45.318482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf7f2d-4f2c-4836-aeb9-8e6dfc04d4ca None admin] PATH_INFO: `/v3/projects/31796c851dfa4e61a8e1afef74d31165/groups/8c2f2e2765a54038bbe175747ee30305/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:45.324126 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf7f2d-4f2c-4836-aeb9-8e6dfc04d4ca None admin] RBAC: Authorizing `identity:create_grant(project_id=31796c851dfa4e61a8e1afef74d31165, group_id=8c2f2e2765a54038bbe175747ee30305, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:45.325775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf7f2d-4f2c-4836-aeb9-8e6dfc04d4ca None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:45.347625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 201/418] 10.222.0.18 () {58 vars in 1442 bytes} [Mon Aug 29 20:19:45 2022] PUT /identity/v3/projects/31796c851dfa4e61a8e1afef74d31165/groups/8c2f2e2765a54038bbe175747ee30305/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:19:45.457798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c61513-ae5f-46f8-b6b4-7945f0628c72 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:45.505645 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30441f66-1bd2-4b4e-b94e-984300c46c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:45.506615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30441f66-1bd2-4b4e-b94e-984300c46c3d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:45.507039 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30441f66-1bd2-4b4e-b94e-984300c46c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:45.507453 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30441f66-1bd2-4b4e-b94e-984300c46c3d None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:45.508312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30441f66-1bd2-4b4e-b94e-984300c46c3d None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:45.508755 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30441f66-1bd2-4b4e-b94e-984300c46c3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:45.510348 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30441f66-1bd2-4b4e-b94e-984300c46c3d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:45.520529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 218/419] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:45.529196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf7f2d-4f2c-4836-aeb9-8e6dfc04d4ca None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:45.642022 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:45.649368 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf7f2d-4f2c-4836-aeb9-8e6dfc04d4ca None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:45.657509 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3b8a3-f8d6-4e26-91c7-174a3f5d3898 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:45.657793 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3b8a3-f8d6-4e26-91c7-174a3f5d3898 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:45.658061 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3b8a3-f8d6-4e26-91c7-174a3f5d3898 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:45.659632 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 219/420] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:45.665208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583db9ad-2026-4fe5-a96a-146a43007faa None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:45.665497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583db9ad-2026-4fe5-a96a-146a43007faa None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:45.665775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583db9ad-2026-4fe5-a96a-146a43007faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:45.692741 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72593d48-eb99-4fd7-b249-a5e26e5d5a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:45.692938 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-583db9ad-2026-4fe5-a96a-146a43007faa None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:45.693778 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72593d48-eb99-4fd7-b249-a5e26e5d5a86 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:45.694070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-583db9ad-2026-4fe5-a96a-146a43007faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:45.694493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72593d48-eb99-4fd7-b249-a5e26e5d5a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:45.694948 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72593d48-eb99-4fd7-b249-a5e26e5d5a86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:45.699299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:45.794761 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 220/421] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:19:45.810471 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72593d48-eb99-4fd7-b249-a5e26e5d5a86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:45.811965 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72593d48-eb99-4fd7-b249-a5e26e5d5a86 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:45.871665 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682d157-5eb8-43bd-a00b-684938dcbad5 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:45.872136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682d157-5eb8-43bd-a00b-684938dcbad5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:45.872508 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682d157-5eb8-43bd-a00b-684938dcbad5 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:45.877439 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 221/422] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:19:45.884517 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99ef49-c694-4b7b-8ff2-110873b0f6e5 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:45.884890 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99ef49-c694-4b7b-8ff2-110873b0f6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:45.885235 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99ef49-c694-4b7b-8ff2-110873b0f6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:45.921075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-df99ef49-c694-4b7b-8ff2-110873b0f6e5 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:45.922196 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-df99ef49-c694-4b7b-8ff2-110873b0f6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:45.956299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:45.969063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 202/423] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:45.976929 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72593d48-eb99-4fd7-b249-a5e26e5d5a86 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:46.064801 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 222/424] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:19:46.078333 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72593d48-eb99-4fd7-b249-a5e26e5d5a86 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:46.093043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df99ef49-c694-4b7b-8ff2-110873b0f6e5 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:46.134829 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856082be-6319-44c2-b8a7-27317b086395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:46.135479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856082be-6319-44c2-b8a7-27317b086395 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:46.135755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856082be-6319-44c2-b8a7-27317b086395 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:46.136047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856082be-6319-44c2-b8a7-27317b086395 None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:46.138830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856082be-6319-44c2-b8a7-27317b086395 None admin] RBAC: Authorizing `identity:get_project(project_id=0370c388ce4e42d2ad10ebca467e24dc)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:46.140319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856082be-6319-44c2-b8a7-27317b086395 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:46.144395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 203/425] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:45 2022] GET /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:46.147664 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856082be-6319-44c2-b8a7-27317b086395 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:46.228404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:46.254043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:46.271196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856082be-6319-44c2-b8a7-27317b086395 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:46.319047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12e347-185a-4cc6-b946-03460e92b201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:46.319700 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12e347-185a-4cc6-b946-03460e92b201 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:46.320016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12e347-185a-4cc6-b946-03460e92b201 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:46.320296 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12e347-185a-4cc6-b946-03460e92b201 None admin] PATH_INFO: `/v3/groups/80832b8b4bff430eb00f2ad1f2236c03` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:46.321888 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12e347-185a-4cc6-b946-03460e92b201 None admin] RBAC: Authorizing `identity:get_group(group_id=80832b8b4bff430eb00f2ad1f2236c03)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:46.324372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12e347-185a-4cc6-b946-03460e92b201 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:46.326442 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 204/426] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 29 20:19:46 2022] GET /identity/v3/groups/80832b8b4bff430eb00f2ad1f2236c03 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:46.332453 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12e347-185a-4cc6-b946-03460e92b201 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:46.365942 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df99ef49-c694-4b7b-8ff2-110873b0f6e5 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:46.406625 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61a651-3f88-401e-b5aa-e74ba8ff6ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:46.407272 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61a651-3f88-401e-b5aa-e74ba8ff6ffb None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:46.407554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61a651-3f88-401e-b5aa-e74ba8ff6ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:46.407830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61a651-3f88-401e-b5aa-e74ba8ff6ffb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:46.419081 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61a651-3f88-401e-b5aa-e74ba8ff6ffb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:46.424132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-4f61a651-3f88-401e-b5aa-e74ba8ff6ffb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:46.424132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:46.424132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:46.424132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:46.424132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:46.424132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:46.424132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:46.424132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:46.424132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:46.424132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:46.424132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:46.424132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:46.424132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:46.424132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:46.424132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:46.424132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:46.425012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:46.425012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:46.425012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:46.425012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:46.425012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:46.425012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:46.425012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:46.425012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:46.425012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:46.425012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:46.425012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:46.425012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:46.425012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:46.425012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:46.425012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:46.425012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:46.425794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:46.425794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:46.425794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:46.425794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:46.425794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:46.425794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:46.425794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:46.425794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:46.425794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:46.425794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:46.425794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:46.425794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:46.425794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 223/427] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:19:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:19:46.426428 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61a651-3f88-401e-b5aa-e74ba8ff6ffb None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:46.451165 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12e347-185a-4cc6-b946-03460e92b201 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:46.487792 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:46.498092 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007074d6-7426-4950-b4a8-b81778c885fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:46.499304 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007074d6-7426-4950-b4a8-b81778c885fe None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:46.499684 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007074d6-7426-4950-b4a8-b81778c885fe None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:46.500375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007074d6-7426-4950-b4a8-b81778c885fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:46.501210 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007074d6-7426-4950-b4a8-b81778c885fe None admin] RBAC: Adding query filter params (group.id=80832b8b4bff430eb00f2ad1f2236c03, role.id=89cf5ed93cc642588a54639342c59ea6, scope.project.id=0370c388ce4e42d2ad10ebca467e24dc) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:46.501785 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007074d6-7426-4950-b4a8-b81778c885fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:46.504117 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007074d6-7426-4950-b4a8-b81778c885fe None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:46.512379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 205/428] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Aug 29 20:19:46 2022] GET /identity/v3/role_assignments?group.id=80832b8b4bff430eb00f2ad1f2236c03&scope.project.id=0370c388ce4e42d2ad10ebca467e24dc&role.id=89cf5ed93cc642588a54639342c59ea6 => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:46.513487 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007074d6-7426-4950-b4a8-b81778c885fe None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:46.535581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61a651-3f88-401e-b5aa-e74ba8ff6ffb None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:46.571604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:46.583315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75239e-5100-4ead-9e90-ce063214fdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:46.583996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75239e-5100-4ead-9e90-ce063214fdf7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:46.584336 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75239e-5100-4ead-9e90-ce063214fdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:46.584621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75239e-5100-4ead-9e90-ce063214fdf7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:46.604785 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75239e-5100-4ead-9e90-ce063214fdf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:46.610489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-df75239e-5100-4ead-9e90-ce063214fdf7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:46.610489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:46.610489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:46.610489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:46.610489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:46.610489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:46.610489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:46.610489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:46.610489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:46.610489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:46.610489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:46.610489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:46.610489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:46.610489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:46.610489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:46.610489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:46.612705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:46.612705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:46.612705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:46.612705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:46.612705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:46.612705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:46.612705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:46.612705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:46.612705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:46.612705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:46.612705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:46.612705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:46.612705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:46.612705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:46.612705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:46.612705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:46.614604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:46.614604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:46.614604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:46.614604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:46.614604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:46.614604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:46.614604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:46.614604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:46.614604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:46.614604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:46.614604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 29 20:19:46.614604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:46.614604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 224/429] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:46.623723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75239e-5100-4ead-9e90-ce063214fdf7 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:46.681789 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007074d6-7426-4950-b4a8-b81778c885fe None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:46.744659 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d81c6-2faf-4910-99dd-04bfac90f882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:46.745541 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d81c6-2faf-4910-99dd-04bfac90f882 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:46.745965 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d81c6-2faf-4910-99dd-04bfac90f882 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:46.746395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d81c6-2faf-4910-99dd-04bfac90f882 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:46.747207 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d81c6-2faf-4910-99dd-04bfac90f882 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:46.747652 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d81c6-2faf-4910-99dd-04bfac90f882 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:46.749327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84d81c6-2faf-4910-99dd-04bfac90f882 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:46.758342 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 206/430] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:19:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:46.764842 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d81c6-2faf-4910-99dd-04bfac90f882 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:46.791115 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75239e-5100-4ead-9e90-ce063214fdf7 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:46.829942 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:46.830751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77dba5e-740a-4694-921f-f1f2ce06c567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:46.831682 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dba5e-740a-4694-921f-f1f2ce06c567 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:46.832214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dba5e-740a-4694-921f-f1f2ce06c567 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:46.832706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dba5e-740a-4694-921f-f1f2ce06c567 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:46.833548 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77dba5e-740a-4694-921f-f1f2ce06c567 None admin] RBAC: Adding query filter params (name=member) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:46.834026 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77dba5e-740a-4694-921f-f1f2ce06c567 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:46.835579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77dba5e-740a-4694-921f-f1f2ce06c567 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:46.845629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 225/431] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:19:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:46.853199 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77dba5e-740a-4694-921f-f1f2ce06c567 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:46.892933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d81c6-2faf-4910-99dd-04bfac90f882 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:46.935866 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:46.941995 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da53e92-337d-4a63-9d89-841ed1e38a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:46.942613 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53e92-337d-4a63-9d89-841ed1e38a3a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:46.943087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53e92-337d-4a63-9d89-841ed1e38a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:46.943498 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53e92-337d-4a63-9d89-841ed1e38a3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:47.006579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77dba5e-740a-4694-921f-f1f2ce06c567 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:47.035799 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da53e92-337d-4a63-9d89-841ed1e38a3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:47.038070 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da53e92-337d-4a63-9d89-841ed1e38a3a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:47.049817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fd703-3734-49ef-9f1e-384fb31d9f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:47.050941 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fd703-3734-49ef-9f1e-384fb31d9f70 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:47.051657 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fd703-3734-49ef-9f1e-384fb31d9f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:47.052169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fd703-3734-49ef-9f1e-384fb31d9f70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:47.115636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:47.141197 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fd703-3734-49ef-9f1e-384fb31d9f70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:47.142740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fd703-3734-49ef-9f1e-384fb31d9f70 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:47.168966 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 207/432] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:47.170569 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be909094-d553-47f3-b15a-29ec54ca796c None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:47.170861 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be909094-d553-47f3-b15a-29ec54ca796c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:47.171137 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be909094-d553-47f3-b15a-29ec54ca796c None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:47.177551 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 208/433] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:47.178360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be909094-d553-47f3-b15a-29ec54ca796c None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:47.202987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:47.274302 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 226/434] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 20:19:47.276967 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ad870-3836-475e-9f48-961b1de77a2d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:47.277491 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ad870-3836-475e-9f48-961b1de77a2d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:47.277963 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ad870-3836-475e-9f48-961b1de77a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:47.301488 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-682ad870-3836-475e-9f48-961b1de77a2d None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:47.303014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-682ad870-3836-475e-9f48-961b1de77a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:47.325852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be909094-d553-47f3-b15a-29ec54ca796c None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:47.366957 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7a64d-588a-4864-aec8-b67919d7b870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:47.368053 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7a64d-588a-4864-aec8-b67919d7b870 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:47.368322 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7a64d-588a-4864-aec8-b67919d7b870 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:47.368591 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7a64d-588a-4864-aec8-b67919d7b870 None admin] PATH_INFO: `/v3/projects/0370c388ce4e42d2ad10ebca467e24dc` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:47.369722 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7a64d-588a-4864-aec8-b67919d7b870 None admin] RBAC: Authorizing `identity:get_project(project_id=0370c388ce4e42d2ad10ebca467e24dc)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:47.371750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7a64d-588a-4864-aec8-b67919d7b870 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:47.377895 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 209/435] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:47 2022] GET /identity/v3/projects/0370c388ce4e42d2ad10ebca467e24dc => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:47.377895 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7a64d-588a-4864-aec8-b67919d7b870 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:47.387530 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 227/436] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:47.389924 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ad870-3836-475e-9f48-961b1de77a2d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:47.406865 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:47.485571 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:47.510508 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7a64d-588a-4864-aec8-b67919d7b870 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:47.512001 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ad870-3836-475e-9f48-961b1de77a2d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:47.549269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104040be-63f2-42f6-b8b0-449aace27ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:47.550264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104040be-63f2-42f6-b8b0-449aace27ace None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:47.550747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104040be-63f2-42f6-b8b0-449aace27ace None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:47.551234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104040be-63f2-42f6-b8b0-449aace27ace None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:47.552726 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104040be-63f2-42f6-b8b0-449aace27ace None admin] RBAC: Authorizing `identity:get_project(project_id=eddee91f1cf44ccf83ca7cd9419ab306)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:47.554588 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104040be-63f2-42f6-b8b0-449aace27ace None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:47.555101 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f660286-8171-4b89-aeae-c70aaef159de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:47.555742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f660286-8171-4b89-aeae-c70aaef159de None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:47.556053 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f660286-8171-4b89-aeae-c70aaef159de None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:47.556332 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f660286-8171-4b89-aeae-c70aaef159de None admin] PATH_INFO: `/v3/groups/80832b8b4bff430eb00f2ad1f2236c03` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:47.562402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f660286-8171-4b89-aeae-c70aaef159de None admin] RBAC: Authorizing `identity:get_group(group_id=80832b8b4bff430eb00f2ad1f2236c03)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:47.562402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f660286-8171-4b89-aeae-c70aaef159de None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:47.562402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 228/437] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 29 20:19:47 2022] GET /identity/v3/groups/80832b8b4bff430eb00f2ad1f2236c03 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:47.563301 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 210/438] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:47 2022] GET /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:47.564873 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f660286-8171-4b89-aeae-c70aaef159de None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:47.568554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104040be-63f2-42f6-b8b0-449aace27ace None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:47.684054 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:47.695141 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104040be-63f2-42f6-b8b0-449aace27ace None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:47.706115 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f660286-8171-4b89-aeae-c70aaef159de None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:47.737943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3266c-0cab-4d1c-a91b-fb5a85ef0fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:47.738847 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3266c-0cab-4d1c-a91b-fb5a85ef0fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:47.739272 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3266c-0cab-4d1c-a91b-fb5a85ef0fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:47.739688 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3266c-0cab-4d1c-a91b-fb5a85ef0fe6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:47.740771 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3266c-0cab-4d1c-a91b-fb5a85ef0fe6 None admin] RBAC: Adding query filter params (group.id=80832b8b4bff430eb00f2ad1f2236c03, role.id=578c9a17c0fa434f9602f5d2b3648785, scope.project.id=0370c388ce4e42d2ad10ebca467e24dc) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:47.741319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3266c-0cab-4d1c-a91b-fb5a85ef0fe6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:47.743037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3266c-0cab-4d1c-a91b-fb5a85ef0fe6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:47.750785 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd613f1-b5e7-4e4b-b88b-84330366870f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:47.751427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd613f1-b5e7-4e4b-b88b-84330366870f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:47.751708 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd613f1-b5e7-4e4b-b88b-84330366870f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:47.752014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd613f1-b5e7-4e4b-b88b-84330366870f None admin] PATH_INFO: `/v3/groups/80832b8b4bff430eb00f2ad1f2236c03` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:47.753192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd613f1-b5e7-4e4b-b88b-84330366870f None admin] RBAC: Authorizing `identity:get_group(group_id=80832b8b4bff430eb00f2ad1f2236c03)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:47.754694 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd613f1-b5e7-4e4b-b88b-84330366870f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:47.762350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 229/439] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 29 20:19:47 2022] GET /identity/v3/groups/80832b8b4bff430eb00f2ad1f2236c03 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:47.762350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd613f1-b5e7-4e4b-b88b-84330366870f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:47.762704 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 211/440] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Aug 29 20:19:47 2022] GET /identity/v3/role_assignments?group.id=80832b8b4bff430eb00f2ad1f2236c03&scope.project.id=0370c388ce4e42d2ad10ebca467e24dc&role.id=578c9a17c0fa434f9602f5d2b3648785 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:47.764662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad788fc1-cd01-407b-8608-65c6446aa34c None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:47.764662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad788fc1-cd01-407b-8608-65c6446aa34c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:47.764662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad788fc1-cd01-407b-8608-65c6446aa34c None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:47.765849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 212/441] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:19:47.768913 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad788fc1-cd01-407b-8608-65c6446aa34c None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:47.786234 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:47.893144 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad788fc1-cd01-407b-8608-65c6446aa34c None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:47.912499 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd613f1-b5e7-4e4b-b88b-84330366870f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:47.928470 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c518129-6e73-4ad3-a287-cdfa0a71ed72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:47.929083 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c518129-6e73-4ad3-a287-cdfa0a71ed72 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:47.929364 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c518129-6e73-4ad3-a287-cdfa0a71ed72 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:47.929645 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c518129-6e73-4ad3-a287-cdfa0a71ed72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:47.930372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c518129-6e73-4ad3-a287-cdfa0a71ed72 None admin] RBAC: Adding query filter params (group.id=80832b8b4bff430eb00f2ad1f2236c03, role.id=89cf5ed93cc642588a54639342c59ea6, scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:47.930678 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c518129-6e73-4ad3-a287-cdfa0a71ed72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:47.932149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c518129-6e73-4ad3-a287-cdfa0a71ed72 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:47.947960 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 213/442] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Aug 29 20:19:47 2022] GET /identity/v3/role_assignments?group.id=80832b8b4bff430eb00f2ad1f2236c03&scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306&role.id=89cf5ed93cc642588a54639342c59ea6 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:47.947960 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717ed4f-e875-4a37-854c-af6f3d2afb8b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:47.947960 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717ed4f-e875-4a37-854c-af6f3d2afb8b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:47.947960 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717ed4f-e875-4a37-854c-af6f3d2afb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:47.962044 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-8717ed4f-e875-4a37-854c-af6f3d2afb8b None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:47.963349 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8717ed4f-e875-4a37-854c-af6f3d2afb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:47.973571 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f96958-028a-4f52-9d99-3596d560447b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:47.974227 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f96958-028a-4f52-9d99-3596d560447b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:47.974505 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f96958-028a-4f52-9d99-3596d560447b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:47.974777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f96958-028a-4f52-9d99-3596d560447b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:47.984738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:47.992574 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f96958-028a-4f52-9d99-3596d560447b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:47.997695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-e2f96958-028a-4f52-9d99-3596d560447b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:47.997695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:47.997695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:47.997695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:47.997695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:47.997695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:47.997695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:47.997695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:47.997695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:47.997695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:47.997695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:47.997695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:47.997695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:47.997695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:47.997695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:47.997695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:47.998893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:47.998893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:47.998893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:47.998893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:47.998893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:47.998893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:47.998893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:47.998893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:47.998893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:47.998893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:47.998893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:47.998893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:47.998893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:47.998893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:47.998893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:47.998893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:48.000886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:48.000886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:48.000886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:48.000886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:48.000886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:48.000886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:48.000886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:48.000886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:48.000886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:48.000886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:48.000886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:48.000886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:48.000886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 230/443] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:19:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:48.005167 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f96958-028a-4f52-9d99-3596d560447b None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:48.064106 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 214/444] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:48.115784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:48.143886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f96958-028a-4f52-9d99-3596d560447b None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:48.181925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84071f-b878-4baf-afdd-6412612017e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:48.182542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84071f-b878-4baf-afdd-6412612017e6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:48.182827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84071f-b878-4baf-afdd-6412612017e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:48.183660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84071f-b878-4baf-afdd-6412612017e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:48.184388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84071f-b878-4baf-afdd-6412612017e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:48.184696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84071f-b878-4baf-afdd-6412612017e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:48.186087 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84071f-b878-4baf-afdd-6412612017e6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:48.196098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 231/445] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:19:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:48.201447 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717ed4f-e875-4a37-854c-af6f3d2afb8b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:48.315693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717ed4f-e875-4a37-854c-af6f3d2afb8b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:48.339157 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84071f-b878-4baf-afdd-6412612017e6 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:48.353044 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223954a-7f06-42da-b2f9-76e8e428f39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:48.353669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223954a-7f06-42da-b2f9-76e8e428f39b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:48.353952 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223954a-7f06-42da-b2f9-76e8e428f39b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:48.354502 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223954a-7f06-42da-b2f9-76e8e428f39b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:48.378256 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:48.439592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a223954a-7f06-42da-b2f9-76e8e428f39b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:48.441032 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a223954a-7f06-42da-b2f9-76e8e428f39b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:48.465089 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84071f-b878-4baf-afdd-6412612017e6 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:48.498911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100928d-1894-4685-8657-842daef7b330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:48.499540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100928d-1894-4685-8657-842daef7b330 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:48.499822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100928d-1894-4685-8657-842daef7b330 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:48.500125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100928d-1894-4685-8657-842daef7b330 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:48.506644 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:48.510727 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100928d-1894-4685-8657-842daef7b330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:48.516192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-0100928d-1894-4685-8657-842daef7b330 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:19:48.516192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:48.516192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:48.516192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:48.516192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:48.516192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:48.516192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:48.516192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:48.516192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:48.516192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:48.516192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:48.516192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:48.516192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:48.516192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:48.516192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:48.516192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:48.517246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:48.517246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:48.517246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:48.517246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:48.517246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:48.517246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:48.517246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:48.517246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:48.517246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:48.517246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:48.517246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:48.517246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:48.517246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:48.517246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:48.517246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:48.517246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:48.518211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:48.518211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:48.518211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:48.518211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:48.518211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:48.518211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:48.518211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:48.518211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:48.518211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:48.518211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:48.518211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:19:48.518211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:48.520074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 232/446] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:19:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:19:48.522683 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100928d-1894-4685-8657-842daef7b330 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:48.541073 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 215/447] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 20:19:48.546524 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223954a-7f06-42da-b2f9-76e8e428f39b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:48.594596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:48.625379 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100928d-1894-4685-8657-842daef7b330 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:48.643598 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223954a-7f06-42da-b2f9-76e8e428f39b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:48.658892 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8eb28-f8f2-498b-a808-65402801a09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:48.659601 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8eb28-f8f2-498b-a808-65402801a09e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:48.659979 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8eb28-f8f2-498b-a808-65402801a09e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:48.660265 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8eb28-f8f2-498b-a808-65402801a09e None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:48.660937 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8eb28-f8f2-498b-a808-65402801a09e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:48.661243 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8eb28-f8f2-498b-a808-65402801a09e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:48.662907 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8eb28-f8f2-498b-a808-65402801a09e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:48.671049 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 233/448] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:48.676446 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8eb28-f8f2-498b-a808-65402801a09e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:48.680941 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e65171-2830-438f-8fce-c862d4a8c9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:48.681568 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e65171-2830-438f-8fce-c862d4a8c9ed None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:48.681844 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e65171-2830-438f-8fce-c862d4a8c9ed None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:48.682116 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e65171-2830-438f-8fce-c862d4a8c9ed None admin] PATH_INFO: `/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:48.683767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e65171-2830-438f-8fce-c862d4a8c9ed None admin] RBAC: Authorizing `identity:get_project(project_id=eddee91f1cf44ccf83ca7cd9419ab306)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:48.685260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e65171-2830-438f-8fce-c862d4a8c9ed None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:48.686870 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 216/449] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:48 2022] GET /identity/v3/projects/eddee91f1cf44ccf83ca7cd9419ab306 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:48.692214 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e65171-2830-438f-8fce-c862d4a8c9ed None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:48.737936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:48.780425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8eb28-f8f2-498b-a808-65402801a09e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:48.790929 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e65171-2830-438f-8fce-c862d4a8c9ed None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:48.808894 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:48.813914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461b299-cf13-4951-b58e-4c4eba3ab71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:48.814530 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461b299-cf13-4951-b58e-4c4eba3ab71d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:48.814806 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461b299-cf13-4951-b58e-4c4eba3ab71d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:48.815091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461b299-cf13-4951-b58e-4c4eba3ab71d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:48.825894 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8311b56-b47c-4669-be20-f00c338874a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:48.826519 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8311b56-b47c-4669-be20-f00c338874a7 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:48.826795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8311b56-b47c-4669-be20-f00c338874a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:48.827066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8311b56-b47c-4669-be20-f00c338874a7 None admin] PATH_INFO: `/v3/groups/80832b8b4bff430eb00f2ad1f2236c03` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:48.828201 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8311b56-b47c-4669-be20-f00c338874a7 None admin] RBAC: Authorizing `identity:get_group(group_id=80832b8b4bff430eb00f2ad1f2236c03)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:48.829629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8311b56-b47c-4669-be20-f00c338874a7 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:48.831374 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 217/450] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 29 20:19:48 2022] GET /identity/v3/groups/80832b8b4bff430eb00f2ad1f2236c03 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:48.836867 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8311b56-b47c-4669-be20-f00c338874a7 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:48.882376 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461b299-cf13-4951-b58e-4c4eba3ab71d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:48.883826 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461b299-cf13-4951-b58e-4c4eba3ab71d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:48.934564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8311b56-b47c-4669-be20-f00c338874a7 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:48.940532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:48.968754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab1b52-718f-43f7-a364-6c0292cad51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:48.969279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab1b52-718f-43f7-a364-6c0292cad51d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:48.969557 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab1b52-718f-43f7-a364-6c0292cad51d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:48.969828 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab1b52-718f-43f7-a364-6c0292cad51d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:48.970547 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab1b52-718f-43f7-a364-6c0292cad51d None admin] RBAC: Adding query filter params (group.id=80832b8b4bff430eb00f2ad1f2236c03, role.id=578c9a17c0fa434f9602f5d2b3648785, scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:48.970857 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab1b52-718f-43f7-a364-6c0292cad51d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:48.972318 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab1b52-718f-43f7-a364-6c0292cad51d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:48.977995 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 218/451] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Aug 29 20:19:48 2022] GET /identity/v3/role_assignments?group.id=80832b8b4bff430eb00f2ad1f2236c03&scope.project.id=eddee91f1cf44ccf83ca7cd9419ab306&role.id=578c9a17c0fa434f9602f5d2b3648785 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:48.983958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 234/452] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 29 20:19:48.989898 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab1b52-718f-43f7-a364-6c0292cad51d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:49.043383 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:49.096253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab1b52-718f-43f7-a364-6c0292cad51d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:49.131425 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84fd1a-8c56-48fa-9818-76b8d549b2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:49.132091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84fd1a-8c56-48fa-9818-76b8d549b2fc None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:49.132373 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84fd1a-8c56-48fa-9818-76b8d549b2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:49.132647 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84fd1a-8c56-48fa-9818-76b8d549b2fc None admin] PATH_INFO: `/v3/projects/31796c851dfa4e61a8e1afef74d31165` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:49.133777 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84fd1a-8c56-48fa-9818-76b8d549b2fc None admin] RBAC: Authorizing `identity:get_project(project_id=31796c851dfa4e61a8e1afef74d31165)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:49.135210 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d84fd1a-8c56-48fa-9818-76b8d549b2fc None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:49.137500 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 219/453] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:19:48 2022] GET /identity/v3/projects/31796c851dfa4e61a8e1afef74d31165 => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:49.143063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461b299-cf13-4951-b58e-4c4eba3ab71d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:49.244282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461b299-cf13-4951-b58e-4c4eba3ab71d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:49.281086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524d069-43b3-4df1-a2b0-f0e8dfd2d63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:49.281728 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524d069-43b3-4df1-a2b0-f0e8dfd2d63e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:49.282017 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524d069-43b3-4df1-a2b0-f0e8dfd2d63e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:49.282292 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524d069-43b3-4df1-a2b0-f0e8dfd2d63e None admin] PATH_INFO: `/v3/groups/8c2f2e2765a54038bbe175747ee30305` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:49.283396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524d069-43b3-4df1-a2b0-f0e8dfd2d63e None admin] RBAC: Authorizing `identity:get_group(group_id=8c2f2e2765a54038bbe175747ee30305)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:49.284933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d524d069-43b3-4df1-a2b0-f0e8dfd2d63e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:49.286842 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 235/454] 10.222.0.18 () {56 vars in 1180 bytes} [Mon Aug 29 20:19:49 2022] GET /identity/v3/groups/8c2f2e2765a54038bbe175747ee30305 => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:49.293011 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84fd1a-8c56-48fa-9818-76b8d549b2fc None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:49.393908 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84fd1a-8c56-48fa-9818-76b8d549b2fc None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:49.410484 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:49.428156 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406c027-620f-4b8c-8115-5f31576f1cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:49.428766 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406c027-620f-4b8c-8115-5f31576f1cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:49.429047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406c027-620f-4b8c-8115-5f31576f1cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:49.429327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406c027-620f-4b8c-8115-5f31576f1cc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:49.430047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406c027-620f-4b8c-8115-5f31576f1cc3 None admin] RBAC: Adding query filter params (group.id=8c2f2e2765a54038bbe175747ee30305, role.id=322da7bd64024770a64bdb73e5c54719, scope.project.id=31796c851dfa4e61a8e1afef74d31165) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:49.430357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406c027-620f-4b8c-8115-5f31576f1cc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:49.431820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406c027-620f-4b8c-8115-5f31576f1cc3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:49.438881 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 220/455] 10.222.0.18 () {56 vars in 1376 bytes} [Mon Aug 29 20:19:49 2022] GET /identity/v3/role_assignments?group.id=8c2f2e2765a54038bbe175747ee30305&scope.project.id=31796c851dfa4e61a8e1afef74d31165&role.id=322da7bd64024770a64bdb73e5c54719 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:51.970051 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617b260-a5c1-460a-b96a-651ab9e355b4 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:51.971077 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617b260-a5c1-460a-b96a-651ab9e355b4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:51.971566 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617b260-a5c1-460a-b96a-651ab9e355b4 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:51.973507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 236/456] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:51.978632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21260b-7a6f-4584-bf6e-efbd193c823d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:51.979460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21260b-7a6f-4584-bf6e-efbd193c823d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:51.979974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21260b-7a6f-4584-bf6e-efbd193c823d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:52.003198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1b21260b-7a6f-4584-bf6e-efbd193c823d None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:52.004817 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1b21260b-7a6f-4584-bf6e-efbd193c823d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:52.101674 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 221/457] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:52.104680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5f816-b5a8-43a2-9e36-9e2cb592f34d None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:52.104966 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5f816-b5a8-43a2-9e36-9e2cb592f34d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:52.105282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5f816-b5a8-43a2-9e36-9e2cb592f34d None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:52.106661 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 237/458] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:52.107391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fec593-d6d3-4b04-80ad-0f885b13faca None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:52.107924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fec593-d6d3-4b04-80ad-0f885b13faca None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:52.108431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fec593-d6d3-4b04-80ad-0f885b13faca None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:52.110245 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 222/459] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:19:52.112147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82432b3-5553-4ca4-8d3b-a772c6522125 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:52.112433 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82432b3-5553-4ca4-8d3b-a772c6522125 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:52.112711 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82432b3-5553-4ca4-8d3b-a772c6522125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:52.115011 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c5fec-50e8-4350-acd6-5660f8df1786 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:52.115588 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c5fec-50e8-4350-acd6-5660f8df1786 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:52.116092 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c5fec-50e8-4350-acd6-5660f8df1786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:52.133092 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d82432b3-5553-4ca4-8d3b-a772c6522125 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:52.135372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d82432b3-5553-4ca4-8d3b-a772c6522125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:52.148693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-527c5fec-50e8-4350-acd6-5660f8df1786 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:52.150923 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-527c5fec-50e8-4350-acd6-5660f8df1786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:52.169909 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:52.262514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 238/460] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:19:52.279124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 223/461] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:19:52.469559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82432b3-5553-4ca4-8d3b-a772c6522125 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:52.577983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c5fec-50e8-4350-acd6-5660f8df1786 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:52.596910 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82432b3-5553-4ca4-8d3b-a772c6522125 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:52.644794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:52.650890 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc09a41-def4-457d-8f26-24314de31f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:52.652700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc09a41-def4-457d-8f26-24314de31f42 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:52.653729 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc09a41-def4-457d-8f26-24314de31f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:52.654203 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc09a41-def4-457d-8f26-24314de31f42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:52.666680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc09a41-def4-457d-8f26-24314de31f42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:52.668219 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc09a41-def4-457d-8f26-24314de31f42 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:52.684948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-5bc09a41-def4-457d-8f26-24314de31f42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:52.684948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:52.684948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:52.684948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:52.684948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:52.684948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:52.684948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:52.684948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:52.684948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:52.684948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:52.684948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:52.684948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:52.684948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:19:52.684948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:19:52.684948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:19:52.684948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:19:52.685885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:52.685885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:52.685885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:52.685885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:52.685885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:52.685885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:52.685885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:52.685885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:52.685885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:52.685885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:52.685885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:52.685885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:52.685885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:52.685885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:52.685885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:52.685885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:52.685885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:19:52.686738 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:19:52.686738 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:52.686738 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:52.691148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 239/462] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:19:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:52.691148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca750d5-3593-43e7-bc67-3e415b6e62d6 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:52.691148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca750d5-3593-43e7-bc67-3e415b6e62d6 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:52.691148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca750d5-3593-43e7-bc67-3e415b6e62d6 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:52.691487 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 240/463] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:52.691985 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca750d5-3593-43e7-bc67-3e415b6e62d6 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:52.705862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:52.713578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c5fec-50e8-4350-acd6-5660f8df1786 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:52.751466 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4cd04-f4e6-4460-9eda-029dd7bc23ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:52.752392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4cd04-f4e6-4460-9eda-029dd7bc23ea None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:52.752687 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4cd04-f4e6-4460-9eda-029dd7bc23ea None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:52.753359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4cd04-f4e6-4460-9eda-029dd7bc23ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:52.767068 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4cd04-f4e6-4460-9eda-029dd7bc23ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:52.768567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4cd04-f4e6-4460-9eda-029dd7bc23ea None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:52.783820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-e7f4cd04-f4e6-4460-9eda-029dd7bc23ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:52.783820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:52.783820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:52.783820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:52.783820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:52.783820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:52.783820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:52.783820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:52.783820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:52.783820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:52.783820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:52.783820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:52.783820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:19:52.783820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:19:52.783820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:19:52.783820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:19:52.784859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:52.784859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:52.784859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:52.784859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:52.784859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:52.784859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:52.784859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:52.784859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:52.784859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:52.784859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:52.784859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:52.784859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:52.784859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:52.784859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:52.784859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:52.784859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:52.784859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:19:52.785715 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:19:52.785715 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:52.785715 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:52.785715 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 224/464] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:19:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:52.786732 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4cd04-f4e6-4460-9eda-029dd7bc23ea None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:52.802290 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca750d5-3593-43e7-bc67-3e415b6e62d6 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:52.837182 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c61f3a-8fd4-4d65-85b8-42c8f5c978a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:52.843959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c61f3a-8fd4-4d65-85b8-42c8f5c978a3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:52.843959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c61f3a-8fd4-4d65-85b8-42c8f5c978a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:52.843959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c61f3a-8fd4-4d65-85b8-42c8f5c978a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:52.849194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c61f3a-8fd4-4d65-85b8-42c8f5c978a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:52.850640 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c61f3a-8fd4-4d65-85b8-42c8f5c978a3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:52.864372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-89c61f3a-8fd4-4d65-85b8-42c8f5c978a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:52.864372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:52.864372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:52.864372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:52.864372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:52.864372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:52.864372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:52.864372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:52.864372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:52.864372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:52.864372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:52.864372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:52.864372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:19:52.864372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:19:52.864372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:19:52.864372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:19:52.865309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:52.865309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:52.865309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:52.865309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:52.865309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:52.865309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:52.865309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:52.865309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:52.865309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:52.865309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:52.865309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:52.865309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:52.865309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:52.865309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:52.865309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:52.865309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:52.865309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:19:52.866145 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:19:52.866145 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:52.866145 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:52.866145 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 241/465] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:19:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:52.866800 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba76f8c-f40f-45db-8afa-f2180da4da64 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:52.866964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba76f8c-f40f-45db-8afa-f2180da4da64 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:52.867202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba76f8c-f40f-45db-8afa-f2180da4da64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:52.885339 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2ba76f8c-f40f-45db-8afa-f2180da4da64 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:52.887023 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2ba76f8c-f40f-45db-8afa-f2180da4da64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:52.889427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:52.896500 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4cd04-f4e6-4460-9eda-029dd7bc23ea None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:52.933037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82057a-1151-41cf-93fb-fdcdb3502a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:52.933686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82057a-1151-41cf-93fb-fdcdb3502a7e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:52.933969 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82057a-1151-41cf-93fb-fdcdb3502a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:52.934245 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82057a-1151-41cf-93fb-fdcdb3502a7e None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:52.934899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82057a-1151-41cf-93fb-fdcdb3502a7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:52.935202 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82057a-1151-41cf-93fb-fdcdb3502a7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:52.937017 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82057a-1151-41cf-93fb-fdcdb3502a7e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:52.956680 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 225/466] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:19:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:52.957544 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82057a-1151-41cf-93fb-fdcdb3502a7e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:52.963316 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:52.979981 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 242/467] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:52.979981 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba76f8c-f40f-45db-8afa-f2180da4da64 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:53.076459 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82057a-1151-41cf-93fb-fdcdb3502a7e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:53.080809 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba76f8c-f40f-45db-8afa-f2180da4da64 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:53.114947 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2c2fa-6e4c-4d67-b330-6bf5a3b74ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:53.115291 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c50cc4-7c74-46eb-9f55-9a61985bc862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:53.115915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c50cc4-7c74-46eb-9f55-9a61985bc862 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:53.116132 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2c2fa-6e4c-4d67-b330-6bf5a3b74ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:53.116236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c50cc4-7c74-46eb-9f55-9a61985bc862 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:53.116483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c50cc4-7c74-46eb-9f55-9a61985bc862 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:53.116778 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2c2fa-6e4c-4d67-b330-6bf5a3b74ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:53.117135 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c50cc4-7c74-46eb-9f55-9a61985bc862 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:53.117336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2c2fa-6e4c-4d67-b330-6bf5a3b74ba2 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:53.117437 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c50cc4-7c74-46eb-9f55-9a61985bc862 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:53.118191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2c2fa-6e4c-4d67-b330-6bf5a3b74ba2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:53.118678 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2c2fa-6e4c-4d67-b330-6bf5a3b74ba2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:53.118782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c50cc4-7c74-46eb-9f55-9a61985bc862 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:53.120447 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2c2fa-6e4c-4d67-b330-6bf5a3b74ba2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:53.131362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 243/468] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:19:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:53.132005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c50cc4-7c74-46eb-9f55-9a61985bc862 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:53.134512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 226/469] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:19:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:53.143127 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2c2fa-6e4c-4d67-b330-6bf5a3b74ba2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:53.159189 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:53.191444 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:53.240285 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c50cc4-7c74-46eb-9f55-9a61985bc862 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:53.258293 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2c2fa-6e4c-4d67-b330-6bf5a3b74ba2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:53.275686 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f7b5d-0f31-4c50-b711-a83dcc6c4060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:53.276340 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f7b5d-0f31-4c50-b711-a83dcc6c4060 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:53.276624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f7b5d-0f31-4c50-b711-a83dcc6c4060 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:53.276898 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f7b5d-0f31-4c50-b711-a83dcc6c4060 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:53.277570 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f7b5d-0f31-4c50-b711-a83dcc6c4060 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:53.279011 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f7b5d-0f31-4c50-b711-a83dcc6c4060 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:53.298705 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccd266-9f74-49f4-ae18-7a7074896a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:53.299314 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccd266-9f74-49f4-ae18-7a7074896a9d None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:53.299596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccd266-9f74-49f4-ae18-7a7074896a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:53.299965 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ccd266-9f74-49f4-ae18-7a7074896a9d None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:53.300723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 244/470] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 29 20:19:53 2022] POST /identity/v3/users => generated 246 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:19:53.301262 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f7b5d-0f31-4c50-b711-a83dcc6c4060 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:53.312470 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ccd266-9f74-49f4-ae18-7a7074896a9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:53.314023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ccd266-9f74-49f4-ae18-7a7074896a9d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:53.344915 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 227/471] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 29 20:19:53 2022] POST /identity/v3/users => generated 248 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:19:53.353179 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccd266-9f74-49f4-ae18-7a7074896a9d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:53.453376 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f7b5d-0f31-4c50-b711-a83dcc6c4060 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:53.465980 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:53.507638 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479157a-1385-4475-8a69-6089b4f342c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:53.508771 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479157a-1385-4475-8a69-6089b4f342c2 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:53.509230 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479157a-1385-4475-8a69-6089b4f342c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:53.509702 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479157a-1385-4475-8a69-6089b4f342c2 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:53.510564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479157a-1385-4475-8a69-6089b4f342c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:53.512389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479157a-1385-4475-8a69-6089b4f342c2 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:53.512533 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:53.541481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 245/472] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 29 20:19:53 2022] POST /identity/v3/users => generated 248 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:19:53.543434 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c8cef-27ac-4f45-ab2e-4d9085af72f5 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:53.543676 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c8cef-27ac-4f45-ab2e-4d9085af72f5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:53.548049 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8c8cef-27ac-4f45-ab2e-4d9085af72f5 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:53.548049 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 246/473] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:53.552644 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4df0f-a09f-48d7-b7f4-ef00ec7530a4 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:53.552937 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4df0f-a09f-48d7-b7f4-ef00ec7530a4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:53.553214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4df0f-a09f-48d7-b7f4-ef00ec7530a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:53.576722 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ccd266-9f74-49f4-ae18-7a7074896a9d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:53.582382 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-89d4df0f-a09f-48d7-b7f4-ef00ec7530a4 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:53.583620 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-89d4df0f-a09f-48d7-b7f4-ef00ec7530a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:53.619613 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77861ce8-0fcd-427c-9296-2ac4c418cdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:53.620327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77861ce8-0fcd-427c-9296-2ac4c418cdac None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:53.620610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77861ce8-0fcd-427c-9296-2ac4c418cdac None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:53.621461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77861ce8-0fcd-427c-9296-2ac4c418cdac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:53.634385 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77861ce8-0fcd-427c-9296-2ac4c418cdac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:53.636036 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77861ce8-0fcd-427c-9296-2ac4c418cdac None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:53.661866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-77861ce8-0fcd-427c-9296-2ac4c418cdac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:53.661866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:53.661866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:53.661866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:53.661866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:53.661866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:53.661866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:53.661866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:53.661866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:53.661866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:53.661866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:53.661866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:53.661866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:19:53.661866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:19:53.661866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:19:53.661866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:19:53.663066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:53.663066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:53.663066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:53.663066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:53.663066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:53.663066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:53.663066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:53.663066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:53.663066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:53.663066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:53.663066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:53.663066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:53.663066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:53.663066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:53.663066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:53.663066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:53.663066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:19:53.663889 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:19:53.663889 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:53.663889 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:53.664379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 228/474] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:19:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:53.670517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77861ce8-0fcd-427c-9296-2ac4c418cdac None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:53.676396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 247/475] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:53.779786 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77861ce8-0fcd-427c-9296-2ac4c418cdac None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:53.803568 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:53.819007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fdf3ba-9a59-405b-82ad-701866344f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:53.819621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdf3ba-9a59-405b-82ad-701866344f79 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:53.819900 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdf3ba-9a59-405b-82ad-701866344f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:53.820236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fdf3ba-9a59-405b-82ad-701866344f79 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:53.820894 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fdf3ba-9a59-405b-82ad-701866344f79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:53.821194 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fdf3ba-9a59-405b-82ad-701866344f79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:53.822926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fdf3ba-9a59-405b-82ad-701866344f79 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:53.837060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 229/476] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:53.842639 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4df0f-a09f-48d7-b7f4-ef00ec7530a4 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:53.944304 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:53.950915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4df0f-a09f-48d7-b7f4-ef00ec7530a4 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:53.973879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fdf3ba-9a59-405b-82ad-701866344f79 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:53.991589 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5635f3-f056-49b9-bea1-5e54e03ceaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:53.992306 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5635f3-f056-49b9-bea1-5e54e03ceaf3 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:53.992619 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5635f3-f056-49b9-bea1-5e54e03ceaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:53.992901 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5635f3-f056-49b9-bea1-5e54e03ceaf3 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:53.993561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5635f3-f056-49b9-bea1-5e54e03ceaf3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:53.995137 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5635f3-f056-49b9-bea1-5e54e03ceaf3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:54.024105 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 248/477] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 29 20:19:53 2022] POST /identity/v3/users => generated 249 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:19:54.106087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fdf3ba-9a59-405b-82ad-701866344f79 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:54.149577 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add395e2-0ced-4d96-ae0d-5466984b717b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:54.150562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add395e2-0ced-4d96-ae0d-5466984b717b None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:54.151074 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add395e2-0ced-4d96-ae0d-5466984b717b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:54.151578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add395e2-0ced-4d96-ae0d-5466984b717b None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:54.152714 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add395e2-0ced-4d96-ae0d-5466984b717b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:54.154351 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add395e2-0ced-4d96-ae0d-5466984b717b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:54.165016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.sql.core [None req-add395e2-0ced-4d96-ae0d-5466984b717b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 20:19:54.165016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 20:19:54.165016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [parameters: {'id': 'baeb84fac5f24cd083c80ca1260ddc7e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 20:19:54.165016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81397) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 20:19:54.182945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-add395e2-0ced-4d96-ae0d-5466984b717b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 20:19:54.182945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:54.182945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 20:19:54.182945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 20:19:54.182945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 20:19:54.182945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 20:19:54.182945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 20:19:54.182945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 20:19:54.182945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 20:19:54.182945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application conn.query(q) Aug 29 20:19:54.182945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 20:19:54.182945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 20:19:54.182945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 20:19:54.182945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application result.read() Aug 29 20:19:54.182945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 20:19:54.183992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 20:19:54.183992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 20:19:54.183992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 20:19:54.183992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 20:19:54.183992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 20:19:54.183992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 20:19:54.183992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 20:19:54.183992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 20:19:54.183992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:54.183992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 20:19:54.183992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:54.183992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:54.183992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 20:19:54.183992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 20:19:54.183992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 29 20:19:54.183992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return ref.to_dict() Aug 29 20:19:54.183992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 20:19:54.183992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application next(self.gen) Aug 29 20:19:54.184882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 20:19:54.184882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application yield resource Aug 29 20:19:54.184882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 20:19:54.184882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application next(self.gen) Aug 29 20:19:54.184882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 20:19:54.184882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 20:19:54.184882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 20:19:54.184882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 20:19:54.184882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 20:19:54.184882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise self.value Aug 29 20:19:54.184882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 20:19:54.184882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 20:19:54.184882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 20:19:54.184882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application session.commit() Aug 29 20:19:54.184882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 20:19:54.184882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 20:19:54.184882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 20:19:54.185729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 20:19:54.185729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 20:19:54.185729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.session.flush() Aug 29 20:19:54.185729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 20:19:54.185729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 20:19:54.185729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 20:19:54.185729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 20:19:54.185729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 20:19:54.185729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application compat.raise_( Aug 29 20:19:54.185729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 20:19:54.185729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception Aug 29 20:19:54.185729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 20:19:54.185729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 20:19:54.185729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 20:19:54.185729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 20:19:54.185729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 20:19:54.185729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 20:19:54.186587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 20:19:54.186587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 20:19:54.186587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 20:19:54.186587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 20:19:54.186587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 20:19:54.186587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 20:19:54.186587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 20:19:54.186587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 20:19:54.186587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 20:19:54.186587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 20:19:54.186587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 20:19:54.186587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 20:19:54.186587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 20:19:54.186587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 20:19:54.186587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 20:19:54.186587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception Aug 29 20:19:54.187376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 20:19:54.187376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 20:19:54.187376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 20:19:54.187376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 20:19:54.187376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 20:19:54.187376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 20:19:54.187376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 20:19:54.187376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application conn.query(q) Aug 29 20:19:54.187376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 20:19:54.187376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 20:19:54.187376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 20:19:54.187376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application result.read() Aug 29 20:19:54.187376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 20:19:54.187376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 20:19:54.187376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 20:19:54.187376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 20:19:54.189479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 20:19:54.189479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 20:19:54.189479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 20:19:54.189479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 20:19:54.189479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 29 20:19:54.189479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 29 20:19:54.189479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [parameters: {'id': 'baeb84fac5f24cd083c80ca1260ddc7e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 29 20:19:54.189479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 20:19:54.189479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:54.189479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:19:54.189479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:54.189479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:54.189479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:54.189479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:54.189479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:54.189479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:54.191396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:54.191396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:54.191396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:54.191396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:54.191396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:54.191396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:54.191396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 29 20:19:54.191396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 29 20:19:54.191396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:54.191396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:54.191396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 29 20:19:54.191396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 29 20:19:54.191396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 20:19:54.191396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 20:19:54.191396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 29 20:19:54.191396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:54.195105 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 230/478] 10.222.0.18 () {60 vars in 1131 bytes} [Mon Aug 29 20:19:53 2022] POST /identity/v3/roles => generated 144 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:19:54.205117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5635f3-f056-49b9-bea1-5e54e03ceaf3 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:54.317194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5635f3-f056-49b9-bea1-5e54e03ceaf3 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:54.358396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fd19d-e72e-411a-a48b-d1a3c8629eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:54.358977 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fd19d-e72e-411a-a48b-d1a3c8629eda None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:54.359255 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fd19d-e72e-411a-a48b-d1a3c8629eda None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:54.359528 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fd19d-e72e-411a-a48b-d1a3c8629eda None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:54.374554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fd19d-e72e-411a-a48b-d1a3c8629eda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:54.382237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-630fd19d-e72e-411a-a48b-d1a3c8629eda None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:54.382237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:54.382237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:54.382237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:54.382237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:54.382237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:54.382237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:54.382237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:54.382237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:54.382237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:54.382237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:54.382237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:54.382237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:54.382237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:54.382237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:54.382237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:54.383204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:54.383204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:54.383204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:54.383204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:54.383204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:54.383204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:54.383204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:54.383204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:54.383204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:54.383204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:54.383204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:54.383204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:54.383204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:54.383204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:54.383204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:54.383204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:54.384022 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:54.384022 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:54.384022 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:54.384022 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:54.384022 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:54.384022 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:54.384022 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:54.384022 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:54.384022 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:54.384022 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:54.384022 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 29 20:19:54.384022 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:54.385169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 249/479] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:19:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:54.391257 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add395e2-0ced-4d96-ae0d-5466984b717b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:54.455498 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:54.524623 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add395e2-0ced-4d96-ae0d-5466984b717b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:54.579967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df8d5a-fbb8-4a3d-bbdc-454ae9e27f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:54.580595 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df8d5a-fbb8-4a3d-bbdc-454ae9e27f8c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:54.580877 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df8d5a-fbb8-4a3d-bbdc-454ae9e27f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:54.581150 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df8d5a-fbb8-4a3d-bbdc-454ae9e27f8c None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:54.581898 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df8d5a-fbb8-4a3d-bbdc-454ae9e27f8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:54.582133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df8d5a-fbb8-4a3d-bbdc-454ae9e27f8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:54.583515 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df8d5a-fbb8-4a3d-bbdc-454ae9e27f8c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:54.592976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 231/480] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:19:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:55.832219 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed609e-0f65-4ee0-a7ca-913dd83d5fd5 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:55.832726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed609e-0f65-4ee0-a7ca-913dd83d5fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:55.832800 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed609e-0f65-4ee0-a7ca-913dd83d5fd5 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:55.834362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 250/481] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:55.839575 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81517a2-edfa-4c59-9fe2-be5057133a30 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:55.840028 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81517a2-edfa-4c59-9fe2-be5057133a30 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:55.840190 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81517a2-edfa-4c59-9fe2-be5057133a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:55.853796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b39c9d-9a60-481c-8fed-586b73a253af None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:55.854106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b39c9d-9a60-481c-8fed-586b73a253af None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:55.854354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b39c9d-9a60-481c-8fed-586b73a253af None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:55.856479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 251/482] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:19:55.860999 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bf9ea-c4a3-4457-9da3-7e8025510fe3 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:55.861282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bf9ea-c4a3-4457-9da3-7e8025510fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:55.861553 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bf9ea-c4a3-4457-9da3-7e8025510fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:55.878066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b81517a2-edfa-4c59-9fe2-be5057133a30 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:55.879302 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b81517a2-edfa-4c59-9fe2-be5057133a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:55.888168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-1f8bf9ea-c4a3-4457-9da3-7e8025510fe3 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:55.889367 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1f8bf9ea-c4a3-4457-9da3-7e8025510fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:55.904278 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:56.007603 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 232/483] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:56.020252 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 252/484] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:56.281968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131ad91-b82c-47d4-9336-e2af4849c50b None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:56.282325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131ad91-b82c-47d4-9336-e2af4849c50b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:56.282527 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131ad91-b82c-47d4-9336-e2af4849c50b None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:56.284164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 233/485] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:56.289331 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d42b8b-6f95-4e50-bf80-79573b992436 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:56.289610 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d42b8b-6f95-4e50-bf80-79573b992436 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:56.289883 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d42b8b-6f95-4e50-bf80-79573b992436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:56.311638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131ad91-b82c-47d4-9336-e2af4849c50b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:56.322819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d9d42b8b-6f95-4e50-bf80-79573b992436 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:56.324039 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d9d42b8b-6f95-4e50-bf80-79573b992436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:56.375544 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:56.414015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 253/486] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:56.414943 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d42b8b-6f95-4e50-bf80-79573b992436 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:56.445643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131ad91-b82c-47d4-9336-e2af4849c50b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:56.486856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16724223-3036-4ddf-bc8c-160367e5d3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:56.487876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16724223-3036-4ddf-bc8c-160367e5d3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:56.488481 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16724223-3036-4ddf-bc8c-160367e5d3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:56.488953 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16724223-3036-4ddf-bc8c-160367e5d3a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:56.507967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16724223-3036-4ddf-bc8c-160367e5d3a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:56.509376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-16724223-3036-4ddf-bc8c-160367e5d3a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:19:56.509376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:56.509376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:56.509376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:56.509376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:56.509376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:56.509376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:56.509376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:56.509376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:56.509376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:56.509376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:56.509376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:56.509376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:56.509376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:56.509376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:56.509376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:56.510234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:56.510234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:56.510234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:56.510234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:56.510234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:56.510234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:56.510234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:56.510234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:56.510234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:56.510234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:56.510234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:56.510234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:56.510234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:56.510234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:56.510234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:56.510234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:56.511196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:56.511196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:56.511196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:56.511196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:56.511196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:56.511196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:56.511196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:56.511196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:56.511196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:56.511196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:56.511196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:19:56.511196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:56.511196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 234/487] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:19:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:19:56.513677 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:56.525591 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16724223-3036-4ddf-bc8c-160367e5d3a8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:56.575694 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d42b8b-6f95-4e50-bf80-79573b992436 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:56.656106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ff055-e98c-48a4-b89a-9c93ebbc2991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:56.658226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ff055-e98c-48a4-b89a-9c93ebbc2991 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:56.658690 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ff055-e98c-48a4-b89a-9c93ebbc2991 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:56.659002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ff055-e98c-48a4-b89a-9c93ebbc2991 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:56.689047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ff055-e98c-48a4-b89a-9c93ebbc2991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:56.694284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-815ff055-e98c-48a4-b89a-9c93ebbc2991 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:19:56.694284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:56.694284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:56.694284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:56.694284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:56.694284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:56.694284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:56.694284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:56.694284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:56.694284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:56.694284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:56.694284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:56.694284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:56.694284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:56.694284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:56.694284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:56.695202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:56.695202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:56.695202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:56.695202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:56.695202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:56.695202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:56.695202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:56.695202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:56.695202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:56.695202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:56.695202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:56.695202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:56.695202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:56.695202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:56.695202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:56.695202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:56.696015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:56.696015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:56.696015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:56.696015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:56.696015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:56.696015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:56.696015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:56.696015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:56.696015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:56.696015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:56.696015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:19:56.696015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:56.702906 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 254/488] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:19:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:56.709149 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ff055-e98c-48a4-b89a-9c93ebbc2991 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:56.740595 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:56.745208 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16724223-3036-4ddf-bc8c-160367e5d3a8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:56.798629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6f870-0756-449a-a131-117bc119eded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:56.799672 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6f870-0756-449a-a131-117bc119eded None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:56.800192 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6f870-0756-449a-a131-117bc119eded None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:56.800657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6f870-0756-449a-a131-117bc119eded None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:56.801516 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d6f870-0756-449a-a131-117bc119eded None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:56.802007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d6f870-0756-449a-a131-117bc119eded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:56.803576 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d6f870-0756-449a-a131-117bc119eded None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:56.813657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 235/489] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:19:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:56.814816 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6f870-0756-449a-a131-117bc119eded None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:56.839053 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ff055-e98c-48a4-b89a-9c93ebbc2991 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:56.877772 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:56.886223 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714cdca-0048-4b1f-953b-f893ca9dba83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:56.886986 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714cdca-0048-4b1f-953b-f893ca9dba83 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:56.887354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714cdca-0048-4b1f-953b-f893ca9dba83 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:56.887654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714cdca-0048-4b1f-953b-f893ca9dba83 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:56.888451 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714cdca-0048-4b1f-953b-f893ca9dba83 None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:56.888752 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714cdca-0048-4b1f-953b-f893ca9dba83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:56.890598 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8714cdca-0048-4b1f-953b-f893ca9dba83 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:56.901038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 255/490] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:19:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:19:56.901755 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714cdca-0048-4b1f-953b-f893ca9dba83 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:56.942097 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6f870-0756-449a-a131-117bc119eded None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:56.991166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41442c5b-ecc4-4b01-adeb-f6ac5e5c0e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:56.992202 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41442c5b-ecc4-4b01-adeb-f6ac5e5c0e64 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:56.992657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41442c5b-ecc4-4b01-adeb-f6ac5e5c0e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:56.993245 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41442c5b-ecc4-4b01-adeb-f6ac5e5c0e64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:57.009900 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41442c5b-ecc4-4b01-adeb-f6ac5e5c0e64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:57.019669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-41442c5b-ecc4-4b01-adeb-f6ac5e5c0e64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:19:57.019669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:57.019669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:57.019669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:57.019669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:57.019669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:57.019669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:57.019669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:57.019669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:57.019669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:57.019669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:57.019669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:57.019669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:57.019669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:57.019669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:57.019669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:57.020713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:57.020713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:57.020713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:57.020713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:57.020713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:57.020713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:57.020713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:57.020713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:57.020713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:57.020713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:57.020713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:57.020713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:57.020713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:57.020713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:57.020713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:57.020713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:57.021482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:57.021482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:57.021482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:57.021482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:57.021482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:57.021482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:57.021482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:57.021482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:57.021482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:57.021482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:57.021482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:19:57.021482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:57.022621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 236/491] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:19:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:57.023965 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3302bb-def8-4c01-a72d-67730e71f551 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:57.024425 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3302bb-def8-4c01-a72d-67730e71f551 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:57.024896 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3302bb-def8-4c01-a72d-67730e71f551 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:57.027895 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 237/492] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:57.028955 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3302bb-def8-4c01-a72d-67730e71f551 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:57.046307 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714cdca-0048-4b1f-953b-f893ca9dba83 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:57.060949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:57.092509 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca475c-f137-438f-b595-2547529f0acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:57.093456 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca475c-f137-438f-b595-2547529f0acc None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:57.094104 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca475c-f137-438f-b595-2547529f0acc None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:57.094361 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca475c-f137-438f-b595-2547529f0acc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:57.107487 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca475c-f137-438f-b595-2547529f0acc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:57.109127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca475c-f137-438f-b595-2547529f0acc None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:57.125070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-35ca475c-f137-438f-b595-2547529f0acc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:57.125070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:57.125070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:57.125070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:57.125070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:57.125070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:57.125070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:57.125070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:57.125070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:57.125070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:57.125070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:57.125070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:57.125070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:19:57.125070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:19:57.125070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:19:57.125070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:19:57.126047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:57.126047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:57.126047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:57.126047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:57.126047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:57.126047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:57.126047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:57.126047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:57.126047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:57.126047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:57.126047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:57.126047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:57.126047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:57.126047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:57.126047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:57.126047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:57.126047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:19:57.126876 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:19:57.126876 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:57.126876 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:57.126876 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 256/493] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:19:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:57.127085 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca475c-f137-438f-b595-2547529f0acc None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:57.160451 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3302bb-def8-4c01-a72d-67730e71f551 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:57.181103 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:57.197762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb57d5-3034-48c9-8b31-47e954c66172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:57.198400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb57d5-3034-48c9-8b31-47e954c66172 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:57.198680 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb57d5-3034-48c9-8b31-47e954c66172 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:57.198952 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb57d5-3034-48c9-8b31-47e954c66172 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:57.211057 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb57d5-3034-48c9-8b31-47e954c66172 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:57.212604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb57d5-3034-48c9-8b31-47e954c66172 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:57.230216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-46cb57d5-3034-48c9-8b31-47e954c66172 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:57.230216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:57.230216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:57.230216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:57.230216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:57.230216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:57.230216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:57.230216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:57.230216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:57.230216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:57.230216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:57.230216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:57.230216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:19:57.230216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:19:57.230216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:19:57.230216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:19:57.231212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:57.231212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:57.231212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:57.231212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:57.231212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:57.231212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:57.231212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:57.231212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:57.231212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:57.231212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:57.231212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:57.231212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:57.231212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:57.231212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:57.231212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:57.231212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:57.231212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:19:57.232111 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:19:57.232111 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:57.232111 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:57.232111 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 238/494] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:19:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:57.232587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460bc9a-eacf-4435-af7d-9a9e0922e73b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:57.232806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460bc9a-eacf-4435-af7d-9a9e0922e73b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:57.233090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460bc9a-eacf-4435-af7d-9a9e0922e73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:57.250952 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1460bc9a-eacf-4435-af7d-9a9e0922e73b None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:57.252187 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1460bc9a-eacf-4435-af7d-9a9e0922e73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:57.253345 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca475c-f137-438f-b595-2547529f0acc None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:57.291946 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb9d9c-af31-4c93-8bb8-0aad82b90c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:57.292611 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb9d9c-af31-4c93-8bb8-0aad82b90c2a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:57.292977 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb9d9c-af31-4c93-8bb8-0aad82b90c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:57.293252 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb9d9c-af31-4c93-8bb8-0aad82b90c2a None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:57.293922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb9d9c-af31-4c93-8bb8-0aad82b90c2a None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:57.294222 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb9d9c-af31-4c93-8bb8-0aad82b90c2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:57.295610 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb9d9c-af31-4c93-8bb8-0aad82b90c2a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:57.306345 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 257/495] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:19:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:57.307703 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5cd9d-fbab-4134-934f-00890fb57ffe None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:57.308024 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5cd9d-fbab-4134-934f-00890fb57ffe None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:57.308304 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a5cd9d-fbab-4134-934f-00890fb57ffe None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:57.309900 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 258/496] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:19:57.310735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5cd9d-fbab-4134-934f-00890fb57ffe None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:57.327833 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 239/497] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:57.328231 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460bc9a-eacf-4435-af7d-9a9e0922e73b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:57.330900 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:57.440788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a5cd9d-fbab-4134-934f-00890fb57ffe None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:57.448471 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:57.461947 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460bc9a-eacf-4435-af7d-9a9e0922e73b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:57.485446 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99e7e8-3316-4d91-9127-ed01b2d9ebb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:57.486581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99e7e8-3316-4d91-9127-ed01b2d9ebb9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:57.487020 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99e7e8-3316-4d91-9127-ed01b2d9ebb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:57.487467 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99e7e8-3316-4d91-9127-ed01b2d9ebb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:57.488343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99e7e8-3316-4d91-9127-ed01b2d9ebb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:57.488701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99e7e8-3316-4d91-9127-ed01b2d9ebb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:57.490306 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99e7e8-3316-4d91-9127-ed01b2d9ebb9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:57.503379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2eb617-6222-4b8c-9c7c-933de134a164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:57.504362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eb617-6222-4b8c-9c7c-933de134a164 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:57.504814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eb617-6222-4b8c-9c7c-933de134a164 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:57.505185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 259/498] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:57.505511 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eb617-6222-4b8c-9c7c-933de134a164 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:57.506373 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2eb617-6222-4b8c-9c7c-933de134a164 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:57.506598 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99e7e8-3316-4d91-9127-ed01b2d9ebb9 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:57.507026 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2eb617-6222-4b8c-9c7c-933de134a164 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:57.508669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2eb617-6222-4b8c-9c7c-933de134a164 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:57.523295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 240/499] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:57.524836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb1a48-f4dd-4ad5-8fe5-0c197c75ad85 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:57.525151 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb1a48-f4dd-4ad5-8fe5-0c197c75ad85 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:57.525427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb1a48-f4dd-4ad5-8fe5-0c197c75ad85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:57.545348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b9cb1a48-f4dd-4ad5-8fe5-0c197c75ad85 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:19:57.546536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b9cb1a48-f4dd-4ad5-8fe5-0c197c75ad85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:19:57.628892 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:57.643344 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 241/500] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:19:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:19:57.644797 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb1a48-f4dd-4ad5-8fe5-0c197c75ad85 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:57.655009 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99e7e8-3316-4d91-9127-ed01b2d9ebb9 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:57.707267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc5f11-4aea-4fdc-a776-315286600af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:57.708181 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc5f11-4aea-4fdc-a776-315286600af0 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:57.708495 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc5f11-4aea-4fdc-a776-315286600af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:57.708988 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc5f11-4aea-4fdc-a776-315286600af0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:57.723301 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc5f11-4aea-4fdc-a776-315286600af0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:57.725376 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc5f11-4aea-4fdc-a776-315286600af0 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:57.742261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-dedc5f11-4aea-4fdc-a776-315286600af0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:57.742261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:57.742261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:57.742261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:57.742261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:57.742261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:57.742261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:57.742261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:57.742261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:57.742261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:57.742261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:57.742261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:57.742261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:19:57.742261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:19:57.742261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:19:57.742261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:19:57.743864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:57.743864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:57.743864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:57.743864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:57.743864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:57.743864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:57.743864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:57.743864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:57.743864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:57.743864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:57.743864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:57.743864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:57.743864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:57.743864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:57.743864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:57.743864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:57.743864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:19:57.745525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:19:57.745525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:57.745525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:57.745525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 260/501] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:19:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:57.750347 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc5f11-4aea-4fdc-a776-315286600af0 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:57.783469 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:57.805776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb1a48-f4dd-4ad5-8fe5-0c197c75ad85 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:57.843470 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347fa1d-6e11-425c-9238-094c63890f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:57.844403 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347fa1d-6e11-425c-9238-094c63890f35 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:57.844675 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347fa1d-6e11-425c-9238-094c63890f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:57.844942 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347fa1d-6e11-425c-9238-094c63890f35 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:57.852555 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347fa1d-6e11-425c-9238-094c63890f35 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:57.854015 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e347fa1d-6e11-425c-9238-094c63890f35 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:57.868237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-e347fa1d-6e11-425c-9238-094c63890f35 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:19:57.868237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:57.868237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:57.868237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:57.868237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:57.868237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:57.868237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:57.868237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:57.868237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:57.868237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:57.868237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:57.868237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:57.868237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:19:57.868237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:19:57.868237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:19:57.868237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:19:57.869242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:57.869242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:57.869242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:19:57.869242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:19:57.869242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:19:57.869242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:19:57.869242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:57.869242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:57.869242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:57.869242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:57.869242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:57.869242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:57.869242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:57.869242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:57.869242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:57.869242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:57.869242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:57.869242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:57.870104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:57.870104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:57.870104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:19:57.870104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:19:57.870104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:19:57.870104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:19:57.870104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:19:57.870104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:19:57.870104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:19:57.870104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:57.870104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 242/502] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:57.870653 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347fa1d-6e11-425c-9238-094c63890f35 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:57.885970 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc5f11-4aea-4fdc-a776-315286600af0 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:57.931895 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203cb22-eaa3-423e-9843-759b80355a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:57.932566 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203cb22-eaa3-423e-9843-759b80355a1e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:57.932850 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203cb22-eaa3-423e-9843-759b80355a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:57.933120 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203cb22-eaa3-423e-9843-759b80355a1e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:57.944018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4203cb22-eaa3-423e-9843-759b80355a1e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:57.945495 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4203cb22-eaa3-423e-9843-759b80355a1e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:57.946028 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:57.961416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-4203cb22-eaa3-423e-9843-759b80355a1e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:19:57.961416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:57.961416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:57.961416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:57.961416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:57.961416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:57.961416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:57.961416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:57.961416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:57.961416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:57.961416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:57.961416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:57.961416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:19:57.961416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:19:57.961416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:19:57.961416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:19:57.962343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:57.962343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:57.962343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:19:57.962343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:19:57.962343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:19:57.962343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:19:57.962343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:57.962343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:57.962343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:57.962343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:57.962343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:57.962343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:57.962343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:57.962343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:57.962343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:57.962343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:57.962343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:57.962343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:57.963407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:57.963407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:57.963407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:19:57.963407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:19:57.963407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:19:57.963407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:19:57.963407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:19:57.963407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:19:57.963407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:19:57.963407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:57.963407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 261/503] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Aug 29 20:19:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:57.964419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203cb22-eaa3-423e-9843-759b80355a1e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:58.004073 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347fa1d-6e11-425c-9238-094c63890f35 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:58.041107 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e887fdd-a23c-405c-b458-62cdc82023ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:58.041740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e887fdd-a23c-405c-b458-62cdc82023ec None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:58.042018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e887fdd-a23c-405c-b458-62cdc82023ec None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:58.042291 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e887fdd-a23c-405c-b458-62cdc82023ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:58.052487 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:58.053664 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e887fdd-a23c-405c-b458-62cdc82023ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:19:58.059392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-8e887fdd-a23c-405c-b458-62cdc82023ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:19:58.059392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:58.059392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:58.059392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:58.059392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:58.059392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:58.059392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:58.059392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:58.059392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:58.059392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:58.059392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:58.059392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:58.059392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:19:58.059392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:19:58.059392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:19:58.059392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:19:58.060332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:19:58.060332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:19:58.060332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:19:58.060332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:19:58.060332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:58.060332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:58.060332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:58.060332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:58.060332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:58.060332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:58.060332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:58.060332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:58.060332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:58.060332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:58.060332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:58.060332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:58.061115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:58.061115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:58.061115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:58.061115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:58.061115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:19:58.061115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:19:58.061115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:19:58.061115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:19:58.061115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:19:58.061115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:19:58.061115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:19:58.061115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:58.061115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 243/504] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:19:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:58.062085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e887fdd-a23c-405c-b458-62cdc82023ec None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:58.089044 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203cb22-eaa3-423e-9843-759b80355a1e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:58.124867 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9fe9b-f897-499c-a5d9-9e9cd5e50509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:58.125489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9fe9b-f897-499c-a5d9-9e9cd5e50509 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:58.125766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9fe9b-f897-499c-a5d9-9e9cd5e50509 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:58.126040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9fe9b-f897-499c-a5d9-9e9cd5e50509 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:58.126694 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9fe9b-f897-499c-a5d9-9e9cd5e50509 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:58.126996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9fe9b-f897-499c-a5d9-9e9cd5e50509 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:58.128431 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9fe9b-f897-499c-a5d9-9e9cd5e50509 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:58.141833 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 262/505] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:58.142671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9fe9b-f897-499c-a5d9-9e9cd5e50509 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:58.173580 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e887fdd-a23c-405c-b458-62cdc82023ec None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:58.186328 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:58.208729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9898f1f-852f-4b35-93dc-914bd0f473ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:58.209544 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9898f1f-852f-4b35-93dc-914bd0f473ef None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:58.209889 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9898f1f-852f-4b35-93dc-914bd0f473ef None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:58.210348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9898f1f-852f-4b35-93dc-914bd0f473ef None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:58.211240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9898f1f-852f-4b35-93dc-914bd0f473ef None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:58.211608 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9898f1f-852f-4b35-93dc-914bd0f473ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:58.213058 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9898f1f-852f-4b35-93dc-914bd0f473ef None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:58.248510 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9fe9b-f897-499c-a5d9-9e9cd5e50509 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:58.253253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 244/506] 10.222.0.18 () {56 vars in 1137 bytes} [Mon Aug 29 20:19:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:58.253253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9898f1f-852f-4b35-93dc-914bd0f473ef None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:58.282754 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e5add-9a00-4f7f-8156-aad257580d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:58.283394 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e5add-9a00-4f7f-8156-aad257580d36 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:58.283679 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e5add-9a00-4f7f-8156-aad257580d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:58.283983 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e5add-9a00-4f7f-8156-aad257580d36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:58.296276 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:58.300464 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e5add-9a00-4f7f-8156-aad257580d36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:58.301933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e5add-9a00-4f7f-8156-aad257580d36 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:58.317587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-555e5add-9a00-4f7f-8156-aad257580d36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:58.317587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:58.317587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:58.317587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:58.317587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:58.317587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:58.317587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:58.317587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:58.317587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:58.317587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:58.317587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:58.317587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:58.317587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:19:58.317587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:19:58.317587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:19:58.317587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:19:58.318655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:58.318655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:58.318655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:58.318655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:58.318655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:58.318655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:58.318655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:58.318655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:58.318655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:58.318655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:58.318655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:58.318655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:58.318655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:58.318655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:58.318655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:58.318655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:58.318655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:19:58.319500 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:19:58.319500 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:58.319500 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:58.319500 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 263/507] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:19:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:58.319500 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e5add-9a00-4f7f-8156-aad257580d36 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:58.355286 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9898f1f-852f-4b35-93dc-914bd0f473ef None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:58.389049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee8afd-c5d3-47c5-aa2f-79205c036a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:58.389642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee8afd-c5d3-47c5-aa2f-79205c036a9b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:58.389915 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee8afd-c5d3-47c5-aa2f-79205c036a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:58.390182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee8afd-c5d3-47c5-aa2f-79205c036a9b None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:58.390890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee8afd-c5d3-47c5-aa2f-79205c036a9b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:58.391189 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee8afd-c5d3-47c5-aa2f-79205c036a9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:58.392658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee8afd-c5d3-47c5-aa2f-79205c036a9b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:58.408942 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 245/508] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Aug 29 20:19:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:58.409736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee8afd-c5d3-47c5-aa2f-79205c036a9b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:58.421013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e5add-9a00-4f7f-8156-aad257580d36 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:58.438775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:58.455778 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fa8f5-4e1a-4ef2-83ba-bbde3cfb4a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:58.456505 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fa8f5-4e1a-4ef2-83ba-bbde3cfb4a7f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:58.456783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fa8f5-4e1a-4ef2-83ba-bbde3cfb4a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:58.457050 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fa8f5-4e1a-4ef2-83ba-bbde3cfb4a7f None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:58.457712 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fa8f5-4e1a-4ef2-83ba-bbde3cfb4a7f None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:58.458017 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fa8f5-4e1a-4ef2-83ba-bbde3cfb4a7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:58.459899 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fa8f5-4e1a-4ef2-83ba-bbde3cfb4a7f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:58.468271 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 264/509] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:19:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:58.469024 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fa8f5-4e1a-4ef2-83ba-bbde3cfb4a7f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:58.519782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee8afd-c5d3-47c5-aa2f-79205c036a9b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:58.524149 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:58.554026 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0cb140-6143-4988-a9e8-69e5b3df83da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:58.554782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0cb140-6143-4988-a9e8-69e5b3df83da None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:58.555054 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0cb140-6143-4988-a9e8-69e5b3df83da None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:58.555326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0cb140-6143-4988-a9e8-69e5b3df83da None admin] PATH_INFO: `/v3/users/glance` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:58.563064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0cb140-6143-4988-a9e8-69e5b3df83da None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:58.564560 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0cb140-6143-4988-a9e8-69e5b3df83da None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:58.572269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fa8f5-4e1a-4ef2-83ba-bbde3cfb4a7f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:58.584682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-1e0cb140-6143-4988-a9e8-69e5b3df83da None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 20:19:58.584682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:58.584682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:58.584682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:58.584682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:58.584682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:58.584682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:58.584682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:58.584682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:58.584682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:58.584682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:58.584682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:58.584682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:19:58.584682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:19:58.584682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:19:58.584682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:19:58.585794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:58.585794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:58.585794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:19:58.585794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:19:58.585794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:19:58.585794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:19:58.585794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:58.585794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:58.585794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:58.585794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:58.585794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:58.585794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:58.585794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:58.585794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:58.585794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:58.585794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:58.585794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:58.585794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:58.586739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:58.586739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:58.586739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:19:58.586739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:19:58.586739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:19:58.586739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:19:58.586739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:19:58.586739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:19:58.586739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 20:19:58.586739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:19:58.586739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 246/510] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:19:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:58.587387 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0cb140-6143-4988-a9e8-69e5b3df83da None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:58.612764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0475e2-1060-4f10-8b5a-f9d8fa718225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:58.613377 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0475e2-1060-4f10-8b5a-f9d8fa718225 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:58.613657 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0475e2-1060-4f10-8b5a-f9d8fa718225 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:58.613931 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0475e2-1060-4f10-8b5a-f9d8fa718225 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:58.683159 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:58.683372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0475e2-1060-4f10-8b5a-f9d8fa718225 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:58.684900 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0475e2-1060-4f10-8b5a-f9d8fa718225 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:58.695544 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0cb140-6143-4988-a9e8-69e5b3df83da None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:58.729188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93587358-b1ff-4019-9e11-c7545f7bc7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:58.729715 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93587358-b1ff-4019-9e11-c7545f7bc7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:58.729988 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93587358-b1ff-4019-9e11-c7545f7bc7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:58.730254 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93587358-b1ff-4019-9e11-c7545f7bc7a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:58.730933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93587358-b1ff-4019-9e11-c7545f7bc7a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:58.731339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93587358-b1ff-4019-9e11-c7545f7bc7a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:58.732783 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93587358-b1ff-4019-9e11-c7545f7bc7a2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:58.736826 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:58.744689 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 247/511] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:58.745499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93587358-b1ff-4019-9e11-c7545f7bc7a2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:58.785130 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 265/512] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:58.785999 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0475e2-1060-4f10-8b5a-f9d8fa718225 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:58.848488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93587358-b1ff-4019-9e11-c7545f7bc7a2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:58.882537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9f671-7f23-4431-8340-a0b6838ca821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:58.883146 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9f671-7f23-4431-8340-a0b6838ca821 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:58.883447 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9f671-7f23-4431-8340-a0b6838ca821 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:58.883767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9f671-7f23-4431-8340-a0b6838ca821 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:58.887665 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0475e2-1060-4f10-8b5a-f9d8fa718225 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:58.903486 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:58.923895 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aedb500-b7fb-4911-804b-0dfd78389a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:58.924556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedb500-b7fb-4911-804b-0dfd78389a8d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:58.924839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedb500-b7fb-4911-804b-0dfd78389a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:58.925117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedb500-b7fb-4911-804b-0dfd78389a8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:58.934990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aedb500-b7fb-4911-804b-0dfd78389a8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:58.936496 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aedb500-b7fb-4911-804b-0dfd78389a8d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:58.951468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-7aedb500-b7fb-4911-804b-0dfd78389a8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:58.951468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:58.951468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:58.951468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:58.951468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:58.951468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:58.951468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:58.951468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:58.951468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:58.951468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:58.951468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:58.951468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:58.951468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:19:58.951468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:19:58.951468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:19:58.951468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:19:58.952443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:58.952443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:58.952443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:58.952443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:58.952443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:58.952443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:58.952443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:58.952443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:58.952443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:58.952443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:58.952443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:58.952443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:58.952443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:58.952443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:58.952443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:58.952443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:58.952443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:19:58.953260 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:19:58.953260 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:58.953260 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:58.953260 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 266/513] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:19:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:58.953465 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9f671-7f23-4431-8340-a0b6838ca821 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:58.953858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aedb500-b7fb-4911-804b-0dfd78389a8d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:58.954424 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9f671-7f23-4431-8340-a0b6838ca821 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:58.959172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:59.055556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aedb500-b7fb-4911-804b-0dfd78389a8d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:59.063239 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 248/514] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:59.063239 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9f671-7f23-4431-8340-a0b6838ca821 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:59.089043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e345b-f907-49c5-a178-bb0844c91978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:59.089656 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e345b-f907-49c5-a178-bb0844c91978 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:59.090026 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e345b-f907-49c5-a178-bb0844c91978 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:59.090241 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e345b-f907-49c5-a178-bb0844c91978 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:59.090979 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e345b-f907-49c5-a178-bb0844c91978 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:59.091281 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e345b-f907-49c5-a178-bb0844c91978 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:59.092735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e345b-f907-49c5-a178-bb0844c91978 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:59.107302 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:59.132937 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 267/515] 10.222.0.18 () {56 vars in 1137 bytes} [Mon Aug 29 20:19:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:59.133779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e345b-f907-49c5-a178-bb0844c91978 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:59.158326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9f671-7f23-4431-8340-a0b6838ca821 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:59.193172 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29fff5-f006-446e-8719-d9d471f10910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:59.193767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29fff5-f006-446e-8719-d9d471f10910 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:59.194084 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29fff5-f006-446e-8719-d9d471f10910 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:59.194353 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29fff5-f006-446e-8719-d9d471f10910 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:59.194978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb29fff5-f006-446e-8719-d9d471f10910 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:59.195745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:59.196454 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb29fff5-f006-446e-8719-d9d471f10910 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:59.217369 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 249/516] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 29 20:19:59 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:19:59.218455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29fff5-f006-446e-8719-d9d471f10910 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:59.236692 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e345b-f907-49c5-a178-bb0844c91978 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:59.276661 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d2855-bd54-434e-8c58-7c1e6393405f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:59.277341 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d2855-bd54-434e-8c58-7c1e6393405f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:59.277682 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d2855-bd54-434e-8c58-7c1e6393405f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:59.277959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d2855-bd54-434e-8c58-7c1e6393405f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:59.292259 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d2855-bd54-434e-8c58-7c1e6393405f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:59.293798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d2855-bd54-434e-8c58-7c1e6393405f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:59.313965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-db9d2855-bd54-434e-8c58-7c1e6393405f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:59.313965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:59.313965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:59.313965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:59.313965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:59.313965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:59.313965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:59.313965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:59.313965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:59.313965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:59.313965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:59.313965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:59.313965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:19:59.313965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:19:59.313965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:19:59.313965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:19:59.314953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:59.314953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:59.314953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:59.314953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:59.314953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:59.314953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:59.314953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:59.314953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:59.314953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:59.314953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:59.314953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:59.314953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:59.314953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:59.314953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:59.314953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:59.314953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:59.314953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:19:59.315827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:19:59.315827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:59.315827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:59.315827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 268/517] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:59.316068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d2855-bd54-434e-8c58-7c1e6393405f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:59.360951 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29fff5-f006-446e-8719-d9d471f10910 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:59.363309 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:59.402248 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb69bda-d3eb-4438-aeff-afbb474bcaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:59.404012 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb69bda-d3eb-4438-aeff-afbb474bcaab None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:59.404012 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb69bda-d3eb-4438-aeff-afbb474bcaab None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:59.404335 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb69bda-d3eb-4438-aeff-afbb474bcaab None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:59.405601 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb69bda-d3eb-4438-aeff-afbb474bcaab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:59.405786 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb69bda-d3eb-4438-aeff-afbb474bcaab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:59.407764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb69bda-d3eb-4438-aeff-afbb474bcaab None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:59.421628 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 250/518] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:59.422340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb69bda-d3eb-4438-aeff-afbb474bcaab None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:59.446644 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d2855-bd54-434e-8c58-7c1e6393405f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:59.462287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:59.484752 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b6ef2-e6bf-4780-844a-f02de5baa99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:59.485396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b6ef2-e6bf-4780-844a-f02de5baa99e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:59.485677 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b6ef2-e6bf-4780-844a-f02de5baa99e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:59.485952 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b6ef2-e6bf-4780-844a-f02de5baa99e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:59.496884 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b6ef2-e6bf-4780-844a-f02de5baa99e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:59.498366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b6ef2-e6bf-4780-844a-f02de5baa99e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:59.513936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-4a3b6ef2-e6bf-4780-844a-f02de5baa99e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:59.513936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:59.513936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:59.513936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:59.513936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:59.513936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:59.513936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:59.513936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:59.513936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:59.513936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:59.513936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:59.513936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:59.513936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:19:59.513936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:19:59.513936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:19:59.513936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:19:59.515161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:59.515161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:59.515161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:59.515161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:59.515161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:59.515161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:59.515161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:59.515161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:59.515161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:59.515161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:59.515161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:59.515161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:59.515161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:59.515161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:59.515161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:59.515161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:59.515161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:19:59.516215 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:19:59.516215 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:19:59.516215 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:59.519583 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 269/519] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:59.520307 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b6ef2-e6bf-4780-844a-f02de5baa99e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:59.542675 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb69bda-d3eb-4438-aeff-afbb474bcaab None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:59.580521 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52fb7d-5c37-4cdb-8a53-42d7889e4a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:59.581058 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52fb7d-5c37-4cdb-8a53-42d7889e4a01 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:59.581332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52fb7d-5c37-4cdb-8a53-42d7889e4a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:59.581618 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52fb7d-5c37-4cdb-8a53-42d7889e4a01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:59.613897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:59.632352 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b6ef2-e6bf-4780-844a-f02de5baa99e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:59.659672 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52fb7d-5c37-4cdb-8a53-42d7889e4a01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:59.661174 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52fb7d-5c37-4cdb-8a53-42d7889e4a01 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:59.670410 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5a25d-4cf2-46eb-a45d-5fb1c8e7de03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:59.671524 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5a25d-4cf2-46eb-a45d-5fb1c8e7de03 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:59.672017 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5a25d-4cf2-46eb-a45d-5fb1c8e7de03 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:59.672294 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5a25d-4cf2-46eb-a45d-5fb1c8e7de03 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:59.672996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5a25d-4cf2-46eb-a45d-5fb1c8e7de03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:59.673298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5a25d-4cf2-46eb-a45d-5fb1c8e7de03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:59.674688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5a25d-4cf2-46eb-a45d-5fb1c8e7de03 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:59.687652 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 270/520] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:59.688553 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5a25d-4cf2-46eb-a45d-5fb1c8e7de03 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:59.721913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:59.774930 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 251/521] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:19:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:19:59.775782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52fb7d-5c37-4cdb-8a53-42d7889e4a01 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:59.803314 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5a25d-4cf2-46eb-a45d-5fb1c8e7de03 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:59.839362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:59.840132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c48530-c128-40f3-b4e3-8a0f7b76078f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:59.840788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c48530-c128-40f3-b4e3-8a0f7b76078f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:59.841103 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c48530-c128-40f3-b4e3-8a0f7b76078f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:59.841443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c48530-c128-40f3-b4e3-8a0f7b76078f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:59.848787 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c48530-c128-40f3-b4e3-8a0f7b76078f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:59.850239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c48530-c128-40f3-b4e3-8a0f7b76078f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:59.862025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-84c48530-c128-40f3-b4e3-8a0f7b76078f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 20:19:59.862025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:19:59.862025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:19:59.862025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:19:59.862025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:19:59.862025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:19:59.862025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:19:59.862025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:19:59.862025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:19:59.862025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:19:59.862025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:19:59.862025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:19:59.862025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:19:59.862025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:19:59.862025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:19:59.862025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:19:59.862961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:19:59.862961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:19:59.862961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:19:59.862961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:19:59.862961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:19:59.862961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:19:59.862961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:19:59.862961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:19:59.862961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:19:59.862961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:19:59.862961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:19:59.862961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:19:59.862961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:19:59.862961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:19:59.862961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:19:59.862961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:19:59.862961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:19:59.862961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:19:59.863839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:19:59.863839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:19:59.863839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:19:59.863839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:19:59.863839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:19:59.863839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:19:59.863839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:19:59.863839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:19:59.863839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 20:19:59.863839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:19:59.863839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 271/522] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:19:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:19:59.865040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c48530-c128-40f3-b4e3-8a0f7b76078f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:59.888622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52fb7d-5c37-4cdb-8a53-42d7889e4a01 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:19:59.925866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef16d08-2c6c-4473-a077-ce29fc1f4f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:19:59.926833 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef16d08-2c6c-4473-a077-ce29fc1f4f52 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:19:59.927316 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef16d08-2c6c-4473-a077-ce29fc1f4f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:19:59.927653 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef16d08-2c6c-4473-a077-ce29fc1f4f52 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:19:59.928332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef16d08-2c6c-4473-a077-ce29fc1f4f52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:19:59.928630 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef16d08-2c6c-4473-a077-ce29fc1f4f52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:19:59.930041 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef16d08-2c6c-4473-a077-ce29fc1f4f52 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:19:59.945816 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 252/523] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:19:59.945816 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef16d08-2c6c-4473-a077-ce29fc1f4f52 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:19:59.966622 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:19:59.985122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c48530-c128-40f3-b4e3-8a0f7b76078f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:00.021986 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ff654-74db-47e5-ba33-fae154e61e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:00.022727 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ff654-74db-47e5-ba33-fae154e61e19 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:00.023126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ff654-74db-47e5-ba33-fae154e61e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:00.023401 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ff654-74db-47e5-ba33-fae154e61e19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:00.031501 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ff654-74db-47e5-ba33-fae154e61e19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:00.033384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4ff654-74db-47e5-ba33-fae154e61e19 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:00.046267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-9b4ff654-74db-47e5-ba33-fae154e61e19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:00.046267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:00.046267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:00.046267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:00.046267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:00.046267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:00.046267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:00.046267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:00.046267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:00.046267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:00.046267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:00.046267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:00.046267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:00.046267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:00.046267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:00.047386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:00.047386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:00.047386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:00.047386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:00.047386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:00.047386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:00.047386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:00.047386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:00.047386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:00.047386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:00.047386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:00.047386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:00.047386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:00.047386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:00.047386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:00.047386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:00.047386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:00.048257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:00.048257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:00.048257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:00.048257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:00.048257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:00.048257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:00.048257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:00.048257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:00.048257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 272/524] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:19:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:00.048257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ff654-74db-47e5-ba33-fae154e61e19 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:00.069783 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef16d08-2c6c-4473-a077-ce29fc1f4f52 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:00.095062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:00.112828 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f764f-f5bd-4f40-941e-67d97316b12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:00.113937 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f764f-f5bd-4f40-941e-67d97316b12d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:00.114487 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f764f-f5bd-4f40-941e-67d97316b12d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:00.114945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f764f-f5bd-4f40-941e-67d97316b12d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:00.126974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f764f-f5bd-4f40-941e-67d97316b12d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:00.128472 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27f764f-f5bd-4f40-941e-67d97316b12d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:00.143820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-a27f764f-f5bd-4f40-941e-67d97316b12d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:00.143820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:00.143820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:00.143820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:00.143820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:00.143820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:00.143820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:00.143820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:00.143820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:00.143820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:00.143820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:00.143820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:00.143820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:00.143820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:00.143820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:00.143820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:00.144776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:00.144776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:00.144776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:00.144776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:00.144776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:00.144776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:00.144776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:00.144776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:00.144776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:00.144776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:00.144776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:00.144776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:00.144776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:00.144776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:00.144776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:00.144776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:00.144776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:00.150408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:00.150408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:00.150408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:00.150408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 253/525] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:00.150408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f764f-f5bd-4f40-941e-67d97316b12d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:00.164082 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ff654-74db-47e5-ba33-fae154e61e19 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:00.200805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809f5ef-28e4-4a80-94bc-3010172d11ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:00.201717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809f5ef-28e4-4a80-94bc-3010172d11ac None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:00.202170 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809f5ef-28e4-4a80-94bc-3010172d11ac None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:00.202612 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809f5ef-28e4-4a80-94bc-3010172d11ac None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:00.203587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809f5ef-28e4-4a80-94bc-3010172d11ac None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:00.204099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809f5ef-28e4-4a80-94bc-3010172d11ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:00.205706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809f5ef-28e4-4a80-94bc-3010172d11ac None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:00.228318 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 273/526] 10.222.0.18 () {56 vars in 1139 bytes} [Mon Aug 29 20:20:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:00.229524 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809f5ef-28e4-4a80-94bc-3010172d11ac None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:00.234473 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:00.260137 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27f764f-f5bd-4f40-941e-67d97316b12d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:00.299380 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc26651-639b-48f8-8871-dd4bfa24e8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:00.303034 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc26651-639b-48f8-8871-dd4bfa24e8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:00.303034 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc26651-639b-48f8-8871-dd4bfa24e8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:00.303034 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc26651-639b-48f8-8871-dd4bfa24e8d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:00.309637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc26651-639b-48f8-8871-dd4bfa24e8d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:00.311128 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc26651-639b-48f8-8871-dd4bfa24e8d2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:00.328926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-cbc26651-639b-48f8-8871-dd4bfa24e8d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:00.328926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:00.328926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:00.328926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:00.328926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:00.328926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:00.328926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:00.328926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:00.328926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:00.328926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:00.328926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:00.328926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:00.328926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:00.328926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:00.328926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:00.330269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:00.330269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:00.330269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:00.330269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:00.330269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:00.330269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:00.330269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:00.330269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:00.330269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:00.330269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:00.330269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:00.330269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:00.330269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:00.330269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:00.330269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:00.330269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:00.330269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:00.331219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:00.331219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:00.331219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:00.331219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:00.331219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:00.331219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:00.331219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:00.331219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:00.331219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 254/527] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:00.331219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc26651-639b-48f8-8871-dd4bfa24e8d2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:00.364232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809f5ef-28e4-4a80-94bc-3010172d11ac None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:00.380124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:00.408571 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e859a-3860-4a8d-a5bf-2d8272c19e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:00.409571 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e859a-3860-4a8d-a5bf-2d8272c19e73 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:00.410088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e859a-3860-4a8d-a5bf-2d8272c19e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:00.410368 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e859a-3860-4a8d-a5bf-2d8272c19e73 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:00.411074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e859a-3860-4a8d-a5bf-2d8272c19e73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:00.411381 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e859a-3860-4a8d-a5bf-2d8272c19e73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:00.413455 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0e859a-3860-4a8d-a5bf-2d8272c19e73 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:00.437261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 274/528] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:00.438135 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e859a-3860-4a8d-a5bf-2d8272c19e73 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:00.462436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc26651-639b-48f8-8871-dd4bfa24e8d2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:00.499531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129875c9-8f52-4e14-bdec-8462df0c1985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:00.500189 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129875c9-8f52-4e14-bdec-8462df0c1985 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:00.500473 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129875c9-8f52-4e14-bdec-8462df0c1985 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:00.500746 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129875c9-8f52-4e14-bdec-8462df0c1985 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:00.501400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129875c9-8f52-4e14-bdec-8462df0c1985 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:00.501699 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129875c9-8f52-4e14-bdec-8462df0c1985 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:00.503507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129875c9-8f52-4e14-bdec-8462df0c1985 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:00.511781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:00.521428 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 255/529] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:00.521428 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129875c9-8f52-4e14-bdec-8462df0c1985 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:00.556671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e859a-3860-4a8d-a5bf-2d8272c19e73 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:00.596602 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64eeaf-8ef4-4775-a098-98fade9e972a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:00.597349 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64eeaf-8ef4-4775-a098-98fade9e972a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:00.597635 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64eeaf-8ef4-4775-a098-98fade9e972a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:00.597913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64eeaf-8ef4-4775-a098-98fade9e972a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:00.629748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:00.637463 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129875c9-8f52-4e14-bdec-8462df0c1985 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:00.678956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf6e7a-f68d-4419-909e-cccaddf21ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:00.679574 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf6e7a-f68d-4419-909e-cccaddf21ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:00.679858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf6e7a-f68d-4419-909e-cccaddf21ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:00.680164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf6e7a-f68d-4419-909e-cccaddf21ef6 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:00.680867 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf6e7a-f68d-4419-909e-cccaddf21ef6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:00.681408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf6e7a-f68d-4419-909e-cccaddf21ef6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:00.682856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf6e7a-f68d-4419-909e-cccaddf21ef6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:00.683690 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64eeaf-8ef4-4775-a098-98fade9e972a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:00.685410 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64eeaf-8ef4-4775-a098-98fade9e972a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:00.702577 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 256/530] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:00.703437 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf6e7a-f68d-4419-909e-cccaddf21ef6 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:00.760035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:00.810951 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 275/531] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:20:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:20:00.811645 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64eeaf-8ef4-4775-a098-98fade9e972a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:00.820533 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf6e7a-f68d-4419-909e-cccaddf21ef6 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:00.854969 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a8cc3-bb5a-4884-b57f-ef94a111596d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:00.856206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a8cc3-bb5a-4884-b57f-ef94a111596d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:00.856629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a8cc3-bb5a-4884-b57f-ef94a111596d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:00.856949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a8cc3-bb5a-4884-b57f-ef94a111596d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:00.857681 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a8cc3-bb5a-4884-b57f-ef94a111596d None admin] RBAC: Adding query filter params (role.id=04bc48cc5b6842029b93a845d6744985, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=85624cea63af40fda09f57ddf2cd5c3a) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:00.857982 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a8cc3-bb5a-4884-b57f-ef94a111596d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:00.859415 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a8cc3-bb5a-4884-b57f-ef94a111596d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:00.865198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 257/532] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:20:00 2022] GET /identity/v3/role_assignments?user.id=85624cea63af40fda09f57ddf2cd5c3a&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=04bc48cc5b6842029b93a845d6744985 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:00.866452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a8cc3-bb5a-4884-b57f-ef94a111596d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:00.904889 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:00.941501 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64eeaf-8ef4-4775-a098-98fade9e972a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:00.981561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6020bfc-bb7a-4d84-9987-8794c5c35f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:00.982225 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6020bfc-bb7a-4d84-9987-8794c5c35f11 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:00.982476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6020bfc-bb7a-4d84-9987-8794c5c35f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:00.982747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6020bfc-bb7a-4d84-9987-8794c5c35f11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:00.990455 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6020bfc-bb7a-4d84-9987-8794c5c35f11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:00.991924 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6020bfc-bb7a-4d84-9987-8794c5c35f11 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:01.004629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-e6020bfc-bb7a-4d84-9987-8794c5c35f11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:01.004629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:01.004629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:01.004629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:01.004629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:01.004629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:01.004629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:01.004629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:01.004629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:01.004629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:01.004629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:01.004629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:01.004629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:01.004629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:01.004629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:01.005559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:01.005559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:01.005559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:01.005559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:01.005559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:01.005559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:01.005559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:01.005559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:01.005559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:01.005559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:01.005559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:01.005559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:01.005559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:01.005559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:01.005559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:01.005559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:01.005559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:01.006384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:01.006384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:01.006384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:01.006384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:01.006384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:01.006384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:01.006384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:01.006384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:01.006384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 276/533] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:01.006830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6020bfc-bb7a-4d84-9987-8794c5c35f11 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:01.029485 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a8cc3-bb5a-4884-b57f-ef94a111596d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:01.047149 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:01.074528 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29e170-590c-4b62-964a-c2f38b8ede7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:01.075596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29e170-590c-4b62-964a-c2f38b8ede7f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:01.075873 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29e170-590c-4b62-964a-c2f38b8ede7f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:01.076331 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29e170-590c-4b62-964a-c2f38b8ede7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:01.077203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29e170-590c-4b62-964a-c2f38b8ede7f None admin] RBAC: Adding query filter params (role.id=04bc48cc5b6842029b93a845d6744985, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=c0c44754a1c04a2b87081aba6cb509d3) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:01.077550 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29e170-590c-4b62-964a-c2f38b8ede7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:01.079177 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29e170-590c-4b62-964a-c2f38b8ede7f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:01.088004 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 258/534] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:20:00 2022] GET /identity/v3/role_assignments?user.id=c0c44754a1c04a2b87081aba6cb509d3&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=04bc48cc5b6842029b93a845d6744985 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:01.089121 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29e170-590c-4b62-964a-c2f38b8ede7f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:01.198943 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6020bfc-bb7a-4d84-9987-8794c5c35f11 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:01.257275 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8a1a9-ee2d-421d-83ea-4ffbc058cea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:01.258066 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8a1a9-ee2d-421d-83ea-4ffbc058cea4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:01.258349 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8a1a9-ee2d-421d-83ea-4ffbc058cea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:01.258623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8a1a9-ee2d-421d-83ea-4ffbc058cea4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:01.263769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:01.273756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8a1a9-ee2d-421d-83ea-4ffbc058cea4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:01.275336 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8a1a9-ee2d-421d-83ea-4ffbc058cea4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:01.288971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29e170-590c-4b62-964a-c2f38b8ede7f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:01.296824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-bea8a1a9-ee2d-421d-83ea-4ffbc058cea4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:01.296824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:01.296824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:01.296824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:01.296824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:01.296824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:01.296824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:01.296824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:01.296824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:01.296824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:01.296824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:01.296824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:01.296824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:01.296824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:01.296824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:01.296824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:01.297762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:01.297762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:01.297762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:01.297762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:01.297762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:01.297762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:01.297762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:01.297762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:01.297762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:01.297762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:01.297762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:01.297762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:01.297762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:01.297762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:01.297762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:01.297762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:01.297762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:01.298617 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:01.298617 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:01.298617 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:01.298617 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 277/535] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:01.299513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f5443-47b1-420c-a571-0d1999a09d6b None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:01.299801 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f5443-47b1-420c-a571-0d1999a09d6b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:01.300781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f5443-47b1-420c-a571-0d1999a09d6b None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:01.305394 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 278/536] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:01.306235 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f5443-47b1-420c-a571-0d1999a09d6b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:01.348546 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10fb88c-6a42-4801-a832-735fd12cc5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:01.350204 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10fb88c-6a42-4801-a832-735fd12cc5be None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:01.350548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10fb88c-6a42-4801-a832-735fd12cc5be None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:01.350888 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10fb88c-6a42-4801-a832-735fd12cc5be None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:01.351661 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10fb88c-6a42-4801-a832-735fd12cc5be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:01.352057 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10fb88c-6a42-4801-a832-735fd12cc5be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:01.353578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10fb88c-6a42-4801-a832-735fd12cc5be None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:01.373348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 259/537] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:01.375107 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f584ab6-945f-4840-8bb7-88c7a2b4628f None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:01.375878 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f584ab6-945f-4840-8bb7-88c7a2b4628f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:01.376242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f584ab6-945f-4840-8bb7-88c7a2b4628f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:01.395995 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:01.403319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6f584ab6-945f-4840-8bb7-88c7a2b4628f None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:01.404586 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6f584ab6-945f-4840-8bb7-88c7a2b4628f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:01.425730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f5443-47b1-420c-a571-0d1999a09d6b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:01.464000 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bec6fc-d5e2-4b63-9af1-503440c07479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:01.464683 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bec6fc-d5e2-4b63-9af1-503440c07479 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:01.465600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bec6fc-d5e2-4b63-9af1-503440c07479 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:01.466002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bec6fc-d5e2-4b63-9af1-503440c07479 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:01.466958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bec6fc-d5e2-4b63-9af1-503440c07479 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:01.467330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bec6fc-d5e2-4b63-9af1-503440c07479 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:01.468920 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bec6fc-d5e2-4b63-9af1-503440c07479 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:01.477565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 260/538] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:20:01.478442 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f584ab6-945f-4840-8bb7-88c7a2b4628f None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:01.483416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 279/539] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:01.493265 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bec6fc-d5e2-4b63-9af1-503440c07479 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:01.580547 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:01.613828 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f584ab6-945f-4840-8bb7-88c7a2b4628f None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:01.656887 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23918aec-185a-46ce-9adf-131961aab22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:01.657926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23918aec-185a-46ce-9adf-131961aab22c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:01.658393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23918aec-185a-46ce-9adf-131961aab22c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:01.658849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23918aec-185a-46ce-9adf-131961aab22c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:01.660327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23918aec-185a-46ce-9adf-131961aab22c None admin] RBAC: Adding query filter params (role.id=04bc48cc5b6842029b93a845d6744985, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=5d022ed8d2ac43ae816aa396a8e2f17a) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:01.660644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23918aec-185a-46ce-9adf-131961aab22c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:01.662118 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23918aec-185a-46ce-9adf-131961aab22c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:01.681952 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 261/540] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:20:01 2022] GET /identity/v3/role_assignments?user.id=5d022ed8d2ac43ae816aa396a8e2f17a&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=04bc48cc5b6842029b93a845d6744985 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:01.716609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bec6fc-d5e2-4b63-9af1-503440c07479 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:01.771344 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cd569-9db1-48d7-bd17-9aa2a660a01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:01.772026 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cd569-9db1-48d7-bd17-9aa2a660a01e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:01.772314 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cd569-9db1-48d7-bd17-9aa2a660a01e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:01.772596 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cd569-9db1-48d7-bd17-9aa2a660a01e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:01.779375 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:01.784778 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364cd569-9db1-48d7-bd17-9aa2a660a01e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:01.786261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364cd569-9db1-48d7-bd17-9aa2a660a01e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:01.789212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23918aec-185a-46ce-9adf-131961aab22c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:01.803796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-364cd569-9db1-48d7-bd17-9aa2a660a01e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:01.803796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:01.803796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:01.803796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:01.803796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:01.803796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:01.803796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:01.803796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:01.803796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:01.803796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:01.803796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:01.803796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:01.803796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:01.803796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:01.803796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:01.804758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:01.804758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:01.804758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:01.804758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:01.804758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:01.804758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:01.804758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:01.804758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:01.804758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:01.804758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:01.804758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:01.804758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:01.804758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:01.804758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:01.804758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:01.804758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:01.804758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:01.805580 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:01.805580 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:01.805580 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:01.805580 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:01.805580 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:01.805580 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:01.805580 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:01.805580 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:01.805580 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 280/541] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:01.811961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cd569-9db1-48d7-bd17-9aa2a660a01e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:01.943828 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23918aec-185a-46ce-9adf-131961aab22c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:01.979092 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cd569-9db1-48d7-bd17-9aa2a660a01e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:01.988885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c772fb-25ee-40dc-8d22-3651ccb915e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:01.989642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c772fb-25ee-40dc-8d22-3651ccb915e6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:01.990025 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c772fb-25ee-40dc-8d22-3651ccb915e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:01.990367 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c772fb-25ee-40dc-8d22-3651ccb915e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:02.008671 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:02.017653 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c772fb-25ee-40dc-8d22-3651ccb915e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:02.026209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-f9c772fb-25ee-40dc-8d22-3651ccb915e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:02.026209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:02.026209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:02.026209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:02.026209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:02.026209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:02.026209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:02.026209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:02.026209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:02.026209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:02.026209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:02.026209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:02.026209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:02.026209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:02.026209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:02.026209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:02.029135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:02.029135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:02.029135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:02.029135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:02.029135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:02.029135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:02.029135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:02.029135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:02.029135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:02.029135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:02.029135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:02.029135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:02.029135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:02.029135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:02.029135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:02.029135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:02.031016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:02.031016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:02.031016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:02.031016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:02.031016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:02.031016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:02.031016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:02.031016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:02.031016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:02.031016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:02.031016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:02.031016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:02.031016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 262/542] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:20:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:02.036873 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c772fb-25ee-40dc-8d22-3651ccb915e6 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:02.039267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02710d4a-60e5-47c4-81bc-11ae3584f25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:02.039883 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02710d4a-60e5-47c4-81bc-11ae3584f25d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:02.040306 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02710d4a-60e5-47c4-81bc-11ae3584f25d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:02.040581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02710d4a-60e5-47c4-81bc-11ae3584f25d None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:02.041282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02710d4a-60e5-47c4-81bc-11ae3584f25d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:02.041591 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02710d4a-60e5-47c4-81bc-11ae3584f25d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:02.043082 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02710d4a-60e5-47c4-81bc-11ae3584f25d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:02.063779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 281/543] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:02.071584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02710d4a-60e5-47c4-81bc-11ae3584f25d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:02.172227 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c772fb-25ee-40dc-8d22-3651ccb915e6 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:02.175718 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:02.210137 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ab7f6-06e7-4f7b-b6c4-b4c71e603366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:02.210856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ab7f6-06e7-4f7b-b6c4-b4c71e603366 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:02.211142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ab7f6-06e7-4f7b-b6c4-b4c71e603366 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:02.211417 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ab7f6-06e7-4f7b-b6c4-b4c71e603366 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:02.212119 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ab7f6-06e7-4f7b-b6c4-b4c71e603366 None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:02.212425 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ab7f6-06e7-4f7b-b6c4-b4c71e603366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:02.213991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ab7f6-06e7-4f7b-b6c4-b4c71e603366 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:02.223782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 263/544] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:20:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:02.233377 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ab7f6-06e7-4f7b-b6c4-b4c71e603366 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:02.242282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02710d4a-60e5-47c4-81bc-11ae3584f25d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:02.300492 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf7528-903c-423d-80be-919f0d3f574e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:02.301108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf7528-903c-423d-80be-919f0d3f574e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:02.301388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf7528-903c-423d-80be-919f0d3f574e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:02.301662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf7528-903c-423d-80be-919f0d3f574e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:02.302403 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf7528-903c-423d-80be-919f0d3f574e None admin] RBAC: Adding query filter params (role.id=04bc48cc5b6842029b93a845d6744985, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=d51e331a4c9c4f038bdb80f8c5b74e42) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:02.302712 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf7528-903c-423d-80be-919f0d3f574e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:02.304164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf7528-903c-423d-80be-919f0d3f574e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:02.320330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 282/545] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:20:02 2022] GET /identity/v3/role_assignments?user.id=d51e331a4c9c4f038bdb80f8c5b74e42&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=04bc48cc5b6842029b93a845d6744985 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:02.342811 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:02.417325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ab7f6-06e7-4f7b-b6c4-b4c71e603366 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:02.477975 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524cbc6e-9ed8-4e3f-b113-9298fe2558c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:02.478654 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cbc6e-9ed8-4e3f-b113-9298fe2558c7 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:02.478936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cbc6e-9ed8-4e3f-b113-9298fe2558c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:02.479210 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cbc6e-9ed8-4e3f-b113-9298fe2558c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:02.498488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524cbc6e-9ed8-4e3f-b113-9298fe2558c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:02.499866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524cbc6e-9ed8-4e3f-b113-9298fe2558c7 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:02.529043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-524cbc6e-9ed8-4e3f-b113-9298fe2558c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:02.529043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:02.529043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:02.529043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:02.529043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:02.529043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:02.529043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:02.529043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:02.529043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:02.529043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:02.529043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:02.529043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:02.529043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:02.529043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:02.529043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:02.529043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:02.529992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:02.529992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:02.529992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:02.529992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:02.529992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:02.529992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:02.529992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:02.529992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:02.529992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:02.529992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:02.529992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:02.529992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:02.529992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:02.529992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:02.529992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:02.529992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:02.529992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:02.530800 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:02.530800 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:02.530800 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:02.530800 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 264/546] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:02.536217 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf7528-903c-423d-80be-919f0d3f574e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:02.701842 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf7528-903c-423d-80be-919f0d3f574e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:02.731048 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:02.754973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405c8ac-c813-4e28-8b9a-a46c71129339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:02.755524 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405c8ac-c813-4e28-8b9a-a46c71129339 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:02.755811 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405c8ac-c813-4e28-8b9a-a46c71129339 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:02.756114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405c8ac-c813-4e28-8b9a-a46c71129339 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:02.756778 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405c8ac-c813-4e28-8b9a-a46c71129339 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:02.757086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405c8ac-c813-4e28-8b9a-a46c71129339 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:02.758468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405c8ac-c813-4e28-8b9a-a46c71129339 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:02.775125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 283/547] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:02.780876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524cbc6e-9ed8-4e3f-b113-9298fe2558c7 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:02.922061 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524cbc6e-9ed8-4e3f-b113-9298fe2558c7 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:02.948510 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:02.962180 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2bd45-6b11-45c7-8850-a81d7a356da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:02.962817 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2bd45-6b11-45c7-8850-a81d7a356da1 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:02.963103 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2bd45-6b11-45c7-8850-a81d7a356da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:02.963431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2bd45-6b11-45c7-8850-a81d7a356da1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:02.972627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2bd45-6b11-45c7-8850-a81d7a356da1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:02.974220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2bd45-6b11-45c7-8850-a81d7a356da1 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:02.987108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-bba2bd45-6b11-45c7-8850-a81d7a356da1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:20:02.987108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:02.987108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:02.987108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:02.987108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:02.987108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:02.987108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:02.987108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:02.987108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:02.987108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:02.987108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:02.987108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:02.987108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:02.987108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:02.987108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:02.987108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:02.988709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:02.988709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:02.988709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:02.988709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:02.988709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:02.988709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:02.988709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:02.988709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:02.988709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:02.988709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:02.988709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:02.988709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:02.988709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:02.988709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:02.988709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:02.988709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:02.988709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:02.988709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:02.989776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:02.989776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:02.989776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:02.989776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:02.989776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:02.989776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:02.989776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:02.989776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:02.989776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:20:02.989776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:02.989776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 265/548] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:20:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:02.997348 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405c8ac-c813-4e28-8b9a-a46c71129339 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:03.146244 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405c8ac-c813-4e28-8b9a-a46c71129339 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:03.195375 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f43ef1-cc08-480e-9afe-72703771e11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:03.196382 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f43ef1-cc08-480e-9afe-72703771e11e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:03.196868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f43ef1-cc08-480e-9afe-72703771e11e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:03.197382 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f43ef1-cc08-480e-9afe-72703771e11e None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:03.198390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f43ef1-cc08-480e-9afe-72703771e11e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:03.198898 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f43ef1-cc08-480e-9afe-72703771e11e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:03.201107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f43ef1-cc08-480e-9afe-72703771e11e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:03.226532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 284/549] 10.222.0.18 () {56 vars in 1135 bytes} [Mon Aug 29 20:20:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:03.232303 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2bd45-6b11-45c7-8850-a81d7a356da1 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:03.313997 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b4a37-e87f-4199-bcef-85171b9e6730 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:03.314572 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b4a37-e87f-4199-bcef-85171b9e6730 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:03.315102 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b4a37-e87f-4199-bcef-85171b9e6730 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:03.318695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 285/550] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:20:03.323542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd43adb-4bc9-487d-8156-4f4c33b4093e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:03.324850 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd43adb-4bc9-487d-8156-4f4c33b4093e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:03.325338 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd43adb-4bc9-487d-8156-4f4c33b4093e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:03.350781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-cfd43adb-4bc9-487d-8156-4f4c33b4093e None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:03.354252 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-cfd43adb-4bc9-487d-8156-4f4c33b4093e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:03.409235 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2bd45-6b11-45c7-8850-a81d7a356da1 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:03.414629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:03.479714 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 286/551] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:20:03.481688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ef234-5f92-498d-ae76-fa132bbe7d80 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:03.481977 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ef234-5f92-498d-ae76-fa132bbe7d80 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:03.482254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ef234-5f92-498d-ae76-fa132bbe7d80 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:03.483566 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 287/552] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:03.489725 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfe7e9-fce9-4070-af7e-53c071bc3e82 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:03.490008 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfe7e9-fce9-4070-af7e-53c071bc3e82 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:03.490286 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfe7e9-fce9-4070-af7e-53c071bc3e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:03.492432 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dcddef-83f8-42bf-be10-e5173e971211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:03.493039 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dcddef-83f8-42bf-be10-e5173e971211 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:03.493313 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dcddef-83f8-42bf-be10-e5173e971211 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:03.493581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dcddef-83f8-42bf-be10-e5173e971211 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:03.537131 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-07dfe7e9-fce9-4070-af7e-53c071bc3e82 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:03.538579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-07dfe7e9-fce9-4070-af7e-53c071bc3e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:03.583181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:03.627615 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dcddef-83f8-42bf-be10-e5173e971211 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:03.629829 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dcddef-83f8-42bf-be10-e5173e971211 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:03.685678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 288/553] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:20:03.766527 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 266/554] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:20:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:20:03.769467 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfe7e9-fce9-4070-af7e-53c071bc3e82 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:03.773131 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dcddef-83f8-42bf-be10-e5173e971211 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:03.888995 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:03.952389 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dcddef-83f8-42bf-be10-e5173e971211 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:03.972387 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:03.975520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfe7e9-fce9-4070-af7e-53c071bc3e82 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:04.020348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfcd55f-2938-4777-a41b-ced1cd88b6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:04.021340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcd55f-2938-4777-a41b-ced1cd88b6df None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:04.021818 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcd55f-2938-4777-a41b-ced1cd88b6df None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:04.022287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcd55f-2938-4777-a41b-ced1cd88b6df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:04.039539 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfcd55f-2938-4777-a41b-ced1cd88b6df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:04.041836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfcd55f-2938-4777-a41b-ced1cd88b6df None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:04.044668 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca823c7e-4d46-4f50-bc10-71d1272e99aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:04.045624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca823c7e-4d46-4f50-bc10-71d1272e99aa None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:04.046107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca823c7e-4d46-4f50-bc10-71d1272e99aa None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:04.047621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca823c7e-4d46-4f50-bc10-71d1272e99aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:04.066247 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca823c7e-4d46-4f50-bc10-71d1272e99aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:04.073668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-cdfcd55f-2938-4777-a41b-ced1cd88b6df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:04.073668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:04.073668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:04.073668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:04.073668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:04.073668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:04.073668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:04.073668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:04.073668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:04.073668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:04.073668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:04.073668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:04.073668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:04.073668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:04.073668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:04.073668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:04.074725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:04.074725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:04.074725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:04.074725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:04.074725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:04.074725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:04.074725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:04.074725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:04.074725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:04.074725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:04.074725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:04.074725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:04.074725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:04.074725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:04.074725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:04.074725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:04.074725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:04.075579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:04.075579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:04.075579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:04.079880 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 267/555] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:04.080114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-ca823c7e-4d46-4f50-bc10-71d1272e99aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:04.080114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:04.080114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:04.080114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:04.080114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:04.080114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:04.080114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:04.080114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:04.080114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:04.080114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:04.080114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:04.080114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:04.080114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:04.080114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:04.080114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:04.080114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:04.081254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:04.081254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:04.081254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:04.081254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:04.081254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:04.081254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:04.081254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:04.081254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:04.081254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:04.081254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:04.081254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:04.081254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:04.081254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:04.081254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:04.081254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:04.081254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:04.085925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:04.085925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:04.085925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:04.085925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:04.085925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:04.085925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:04.085925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:04.085925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:04.085925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:04.085925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:04.085925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:04.085925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:04.085925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 289/556] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:20:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:04.085925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca823c7e-4d46-4f50-bc10-71d1272e99aa None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:04.086764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfcd55f-2938-4777-a41b-ced1cd88b6df None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:04.196366 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfcd55f-2938-4777-a41b-ced1cd88b6df None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:04.201607 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca823c7e-4d46-4f50-bc10-71d1272e99aa None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:04.223809 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:04.235503 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540865e-ee3d-40d9-99c9-0076c5c45a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:04.236201 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540865e-ee3d-40d9-99c9-0076c5c45a8d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:04.236483 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540865e-ee3d-40d9-99c9-0076c5c45a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:04.236757 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540865e-ee3d-40d9-99c9-0076c5c45a8d None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:04.237412 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540865e-ee3d-40d9-99c9-0076c5c45a8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:04.237717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540865e-ee3d-40d9-99c9-0076c5c45a8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:04.239089 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540865e-ee3d-40d9-99c9-0076c5c45a8d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:04.242449 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f9ea6-aafd-4122-95a4-530b4f94fd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:04.243295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f9ea6-aafd-4122-95a4-530b4f94fd43 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:04.243751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f9ea6-aafd-4122-95a4-530b4f94fd43 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:04.244247 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f9ea6-aafd-4122-95a4-530b4f94fd43 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:04.251555 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 268/557] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:04.252408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540865e-ee3d-40d9-99c9-0076c5c45a8d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:04.261252 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f9ea6-aafd-4122-95a4-530b4f94fd43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:04.267559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-e81f9ea6-aafd-4122-95a4-530b4f94fd43 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:04.267559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:04.267559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:04.267559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:04.267559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:04.267559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:04.267559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:04.267559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:04.267559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:04.267559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:04.267559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:04.267559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:04.267559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:04.267559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:04.267559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:04.267559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:04.269169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:04.269169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:04.269169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:04.269169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:04.269169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:04.269169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:04.269169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:04.269169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:04.269169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:04.269169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:04.269169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:04.269169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:04.269169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:04.269169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:04.269169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:04.269169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:04.270133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:04.270133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:04.270133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:04.270133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:04.270133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:04.270133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:04.270133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:04.270133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:04.270133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:04.270133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:04.270133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:04.270133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:04.272156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 290/558] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:20:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:04.273359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386518b3-2b2f-4c1a-97c8-2e80636b691e None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:04.273647 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386518b3-2b2f-4c1a-97c8-2e80636b691e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:04.273935 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386518b3-2b2f-4c1a-97c8-2e80636b691e None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:04.279764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 291/559] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:04.280577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386518b3-2b2f-4c1a-97c8-2e80636b691e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:04.306935 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:04.368854 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540865e-ee3d-40d9-99c9-0076c5c45a8d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:04.399476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386518b3-2b2f-4c1a-97c8-2e80636b691e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:04.405489 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81725ae2-338f-49f3-972f-733d1cf7c138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:04.406099 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81725ae2-338f-49f3-972f-733d1cf7c138 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:04.406378 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81725ae2-338f-49f3-972f-733d1cf7c138 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:04.406649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81725ae2-338f-49f3-972f-733d1cf7c138 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:04.407307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81725ae2-338f-49f3-972f-733d1cf7c138 None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:04.407863 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81725ae2-338f-49f3-972f-733d1cf7c138 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:04.409008 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81725ae2-338f-49f3-972f-733d1cf7c138 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:04.417733 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 269/560] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:20:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:04.424087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81725ae2-338f-49f3-972f-733d1cf7c138 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:04.439131 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd93142-25a3-43ed-b790-dc56434c1055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:04.439775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd93142-25a3-43ed-b790-dc56434c1055 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:04.440116 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd93142-25a3-43ed-b790-dc56434c1055 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:04.440395 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd93142-25a3-43ed-b790-dc56434c1055 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:04.447181 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd93142-25a3-43ed-b790-dc56434c1055 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:04.448686 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd93142-25a3-43ed-b790-dc56434c1055 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:04.460630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-bdd93142-25a3-43ed-b790-dc56434c1055 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:04.460630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:04.460630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:04.460630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:04.460630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:04.460630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:04.460630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:04.460630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:04.460630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:04.460630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:04.460630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:04.460630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:04.460630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:04.460630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:04.460630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:04.461573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:04.461573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:04.461573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:04.461573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:04.461573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:04.461573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:04.461573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:04.461573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:04.461573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:04.461573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:04.461573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:04.461573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:04.461573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:04.461573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:04.461573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:04.461573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:04.461573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:04.462440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:04.462440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:04.462440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:04.462440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:04.462440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:04.462440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:04.462440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:04.462440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:04.462440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 292/561] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:04.463642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599178a-ae5c-4f1a-8df6-58e269a1257e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:04.463642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599178a-ae5c-4f1a-8df6-58e269a1257e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:04.463642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599178a-ae5c-4f1a-8df6-58e269a1257e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:04.480250 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:04.482407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-0599178a-ae5c-4f1a-8df6-58e269a1257e None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:04.483617 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0599178a-ae5c-4f1a-8df6-58e269a1257e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:04.546759 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81725ae2-338f-49f3-972f-733d1cf7c138 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:04.559265 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 293/562] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:20:04.560148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599178a-ae5c-4f1a-8df6-58e269a1257e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:04.577102 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:04.593435 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9db079-48eb-4441-b542-fb576eac8392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:04.594047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9db079-48eb-4441-b542-fb576eac8392 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:04.594327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9db079-48eb-4441-b542-fb576eac8392 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:04.594658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9db079-48eb-4441-b542-fb576eac8392 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:04.595328 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9db079-48eb-4441-b542-fb576eac8392 None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:04.595683 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9db079-48eb-4441-b542-fb576eac8392 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:04.597169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9db079-48eb-4441-b542-fb576eac8392 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:04.605169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 270/563] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:20:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:04.606518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9db079-48eb-4441-b542-fb576eac8392 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:04.680897 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599178a-ae5c-4f1a-8df6-58e269a1257e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:04.717731 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9db079-48eb-4441-b542-fb576eac8392 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:04.720800 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360094e8-dfb7-4939-b5d1-cb339e68f9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:04.720800 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360094e8-dfb7-4939-b5d1-cb339e68f9af None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:04.720999 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360094e8-dfb7-4939-b5d1-cb339e68f9af None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:04.721273 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360094e8-dfb7-4939-b5d1-cb339e68f9af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:04.728346 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:04.732594 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360094e8-dfb7-4939-b5d1-cb339e68f9af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:04.734152 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360094e8-dfb7-4939-b5d1-cb339e68f9af None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:04.749531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-360094e8-dfb7-4939-b5d1-cb339e68f9af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:04.749531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:04.749531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:04.749531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:04.749531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:04.749531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:04.749531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:04.749531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:04.749531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:04.749531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:04.749531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:04.749531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:04.749531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:04.749531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:04.749531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:04.749531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:04.750480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:04.750480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:04.750480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:04.750480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:04.750480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:04.750480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:04.750480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:04.750480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:04.750480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:04.750480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:04.750480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:04.750480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:04.750480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:04.750480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:04.750480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:04.750480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:04.750480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:04.751350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:04.751350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:04.751350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:04.751350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 294/564] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:04.752018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c5817-3726-4de4-80b5-affe89433410 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:04.752129 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c5817-3726-4de4-80b5-affe89433410 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:04.752342 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c5817-3726-4de4-80b5-affe89433410 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:04.753718 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 295/565] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:04.758118 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c5817-3726-4de4-80b5-affe89433410 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:04.765064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddba4d1-658f-4df4-9f59-b2bcfc9a15b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:04.765959 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddba4d1-658f-4df4-9f59-b2bcfc9a15b0 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:04.766443 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddba4d1-658f-4df4-9f59-b2bcfc9a15b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:04.766914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddba4d1-658f-4df4-9f59-b2bcfc9a15b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:04.768386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddba4d1-658f-4df4-9f59-b2bcfc9a15b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:04.768899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddba4d1-658f-4df4-9f59-b2bcfc9a15b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:04.771438 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddba4d1-658f-4df4-9f59-b2bcfc9a15b0 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:04.789581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 271/566] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:04.789581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddba4d1-658f-4df4-9f59-b2bcfc9a15b0 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:04.830924 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:04.875394 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c5817-3726-4de4-80b5-affe89433410 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:04.889479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddba4d1-658f-4df4-9f59-b2bcfc9a15b0 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:04.909346 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5bf92-ba30-4f01-bfbe-cd681a2f15f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:04.909974 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5bf92-ba30-4f01-bfbe-cd681a2f15f6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:04.910261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5bf92-ba30-4f01-bfbe-cd681a2f15f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:04.910531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5bf92-ba30-4f01-bfbe-cd681a2f15f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:04.921113 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a5bf92-ba30-4f01-bfbe-cd681a2f15f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:04.922556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a5bf92-ba30-4f01-bfbe-cd681a2f15f6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:04.923958 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689f12f-17cd-4f9a-b7de-3f267a26f0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:04.924563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689f12f-17cd-4f9a-b7de-3f267a26f0f0 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:04.924844 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689f12f-17cd-4f9a-b7de-3f267a26f0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:04.925124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689f12f-17cd-4f9a-b7de-3f267a26f0f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:04.925762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689f12f-17cd-4f9a-b7de-3f267a26f0f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:04.926091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689f12f-17cd-4f9a-b7de-3f267a26f0f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:04.927460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689f12f-17cd-4f9a-b7de-3f267a26f0f0 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:04.936564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-76a5bf92-ba30-4f01-bfbe-cd681a2f15f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:04.936564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:04.936564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:04.936564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:04.936564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:04.936564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:04.936564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:04.936564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:04.936564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:04.936564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:04.936564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:04.936564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:04.936564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:04.936564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:04.936564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:04.936564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:04.937687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:04.937687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:04.937687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:04.937687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:04.937687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:04.937687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:04.937687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:04.937687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:04.937687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:04.937687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:04.937687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:04.937687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:04.937687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:04.937687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:04.937687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:04.937687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:04.937687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:04.938744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:04.938744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:04.938744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:04.938744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 296/567] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:04.939016 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2bd56-38f9-4eef-8a0b-9f9f10a84a05 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:04.939096 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2bd56-38f9-4eef-8a0b-9f9f10a84a05 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:04.939326 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2bd56-38f9-4eef-8a0b-9f9f10a84a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:04.945788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 272/568] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:04.945788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689f12f-17cd-4f9a-b7de-3f267a26f0f0 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:04.961407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-92b2bd56-38f9-4eef-8a0b-9f9f10a84a05 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:04.962600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-92b2bd56-38f9-4eef-8a0b-9f9f10a84a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:04.972045 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:05.029527 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 297/569] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:20:05.030396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2bd56-38f9-4eef-8a0b-9f9f10a84a05 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:05.042618 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689f12f-17cd-4f9a-b7de-3f267a26f0f0 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:05.078609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d2cd8-7c2a-4adc-a2e0-b330b24d4290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:05.079214 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d2cd8-7c2a-4adc-a2e0-b330b24d4290 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:05.079493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d2cd8-7c2a-4adc-a2e0-b330b24d4290 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:05.079774 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d2cd8-7c2a-4adc-a2e0-b330b24d4290 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:05.080529 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d2cd8-7c2a-4adc-a2e0-b330b24d4290 None admin] RBAC: Adding query filter params (role.id=04bc48cc5b6842029b93a845d6744985, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=599129b7d8ff47efb145787414ddb80d) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:05.080864 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d2cd8-7c2a-4adc-a2e0-b330b24d4290 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:05.082645 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d2cd8-7c2a-4adc-a2e0-b330b24d4290 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:05.096755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 273/570] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:20:04 2022] GET /identity/v3/role_assignments?user.id=599129b7d8ff47efb145787414ddb80d&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=04bc48cc5b6842029b93a845d6744985 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:05.096755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d2cd8-7c2a-4adc-a2e0-b330b24d4290 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:05.099794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:05.181671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2bd56-38f9-4eef-8a0b-9f9f10a84a05 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:05.203645 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:05.214090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d2cd8-7c2a-4adc-a2e0-b330b24d4290 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:05.235338 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fd14e-ee13-4a9d-9ce4-b2978cf75547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:05.236108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fd14e-ee13-4a9d-9ce4-b2978cf75547 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:05.236457 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fd14e-ee13-4a9d-9ce4-b2978cf75547 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:05.237369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fd14e-ee13-4a9d-9ce4-b2978cf75547 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:05.253457 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fd14e-ee13-4a9d-9ce4-b2978cf75547 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:05.253792 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4574b6-f63f-47ac-9974-1ab27a46b52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:05.254416 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4574b6-f63f-47ac-9974-1ab27a46b52e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:05.254970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4574b6-f63f-47ac-9974-1ab27a46b52e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:05.254970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4574b6-f63f-47ac-9974-1ab27a46b52e None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:05.255612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4574b6-f63f-47ac-9974-1ab27a46b52e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:05.255917 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4574b6-f63f-47ac-9974-1ab27a46b52e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:05.257342 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4574b6-f63f-47ac-9974-1ab27a46b52e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:05.258498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-b47fd14e-ee13-4a9d-9ce4-b2978cf75547 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:05.258498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:05.258498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:05.258498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:05.258498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:05.258498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:05.258498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:05.258498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:05.258498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:05.258498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:05.258498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:05.258498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:05.258498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:05.258498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:05.258498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:05.258498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:05.259365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:05.259365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:05.259365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:05.259365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:05.259365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:05.259365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:05.259365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:05.259365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:05.259365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:05.259365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:05.259365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:05.259365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:05.259365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:05.259365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:05.259365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:05.259365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:05.260229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:05.260229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:05.260229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:05.260229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:05.260229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:05.260229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:05.260229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:05.260229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:05.260229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:05.260229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:05.260229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:05.260229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:05.262646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 298/571] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:20:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:05.265243 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fd14e-ee13-4a9d-9ce4-b2978cf75547 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:05.276288 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 274/572] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:05.277205 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4574b6-f63f-47ac-9974-1ab27a46b52e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:05.388484 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fd14e-ee13-4a9d-9ce4-b2978cf75547 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:05.390278 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:05.396867 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4574b6-f63f-47ac-9974-1ab27a46b52e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:05.429808 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c606b0-9090-4034-8703-bc779b8b1a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:05.430445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c606b0-9090-4034-8703-bc779b8b1a15 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:05.430730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c606b0-9090-4034-8703-bc779b8b1a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:05.430996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c606b0-9090-4034-8703-bc779b8b1a15 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:05.432736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa759f6-5a13-4e32-ab94-3b157ea7a640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:05.433342 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa759f6-5a13-4e32-ab94-3b157ea7a640 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:05.433622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa759f6-5a13-4e32-ab94-3b157ea7a640 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:05.434462 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa759f6-5a13-4e32-ab94-3b157ea7a640 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:05.435128 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa759f6-5a13-4e32-ab94-3b157ea7a640 None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:05.435427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa759f6-5a13-4e32-ab94-3b157ea7a640 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:05.436826 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa759f6-5a13-4e32-ab94-3b157ea7a640 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:05.439527 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c606b0-9090-4034-8703-bc779b8b1a15 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:05.441021 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c606b0-9090-4034-8703-bc779b8b1a15 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:05.449956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 275/573] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:20:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:05.449956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa759f6-5a13-4e32-ab94-3b157ea7a640 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:05.456719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-e5c606b0-9090-4034-8703-bc779b8b1a15 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:20:05.456719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:05.456719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:05.456719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:05.456719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:05.456719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:05.456719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:05.456719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:05.456719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:05.456719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:05.456719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:05.456719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:05.456719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:05.456719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:05.456719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:05.456719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:05.457730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:05.457730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:05.457730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:05.457730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:05.457730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:05.457730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:05.457730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:05.457730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:05.457730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:05.457730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:05.457730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:05.457730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:05.457730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:05.457730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:05.457730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:05.457730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:05.457730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:05.457730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:05.458664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:05.458664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:05.458664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:05.458664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:05.458664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:05.458664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:05.458664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:05.458664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:05.458664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:20:05.458664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:05.458664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 299/574] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:20:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:05.459252 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c606b0-9090-4034-8703-bc779b8b1a15 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:05.472395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:05.560653 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa759f6-5a13-4e32-ab94-3b157ea7a640 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:05.577172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c606b0-9090-4034-8703-bc779b8b1a15 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:05.595486 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc162f-7b25-4360-ae4f-b11699b56259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:05.596147 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc162f-7b25-4360-ae4f-b11699b56259 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:05.596431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc162f-7b25-4360-ae4f-b11699b56259 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:05.596705 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc162f-7b25-4360-ae4f-b11699b56259 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:05.604478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc162f-7b25-4360-ae4f-b11699b56259 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:05.606034 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc162f-7b25-4360-ae4f-b11699b56259 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:05.616806 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f11167-0a08-4450-a157-736723aa98d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:05.617651 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f11167-0a08-4450-a157-736723aa98d5 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:05.617830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-28bc162f-7b25-4360-ae4f-b11699b56259 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:20:05.617830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:05.617830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:05.617830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:05.617830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:05.617830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:05.617830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:05.617830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:05.617830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:05.617830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:05.617830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:05.617830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:05.617830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:05.617830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:05.617830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:05.617830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:05.618884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:05.618884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:05.618884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:05.618884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:05.618884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:05.618884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:05.618884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:05.618884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:05.618884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:05.618884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:05.618884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:05.618884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:05.618884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:05.618884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:05.618884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:05.618884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:05.618884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:05.618884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:05.619822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:05.619822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:05.619822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:05.619822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:05.619822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:05.619822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:05.619822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:05.619822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:05.619822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:20:05.619822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:05.619822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 276/575] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Aug 29 20:20:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:05.620435 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc162f-7b25-4360-ae4f-b11699b56259 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:05.620925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f11167-0a08-4450-a157-736723aa98d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:05.621431 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f11167-0a08-4450-a157-736723aa98d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:05.638629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f11167-0a08-4450-a157-736723aa98d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:05.650845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-05f11167-0a08-4450-a157-736723aa98d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:05.650845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:05.650845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:05.650845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:05.650845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:05.650845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:05.650845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:05.650845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:05.650845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:05.650845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:05.650845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:05.650845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:05.650845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:05.650845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:05.650845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:05.650845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:05.651913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:05.651913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:05.651913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:05.651913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:05.651913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:05.651913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:05.651913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:05.651913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:05.651913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:05.651913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:05.651913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:05.651913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:05.651913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:05.651913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:05.651913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:05.651913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:05.653916 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:05.653916 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:05.653916 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:05.653916 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:05.653916 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:05.653916 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:05.653916 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:05.653916 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:05.653916 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:05.653916 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:05.653916 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:05.653916 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:05.654996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 300/576] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:20:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:05.660106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f11167-0a08-4450-a157-736723aa98d5 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:05.682460 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:05.707013 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:05.733127 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc162f-7b25-4360-ae4f-b11699b56259 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:05.769202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f11167-0a08-4450-a157-736723aa98d5 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:05.769800 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758aed94-f764-472c-ad14-339a37d271c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:05.770433 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758aed94-f764-472c-ad14-339a37d271c5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:05.770716 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758aed94-f764-472c-ad14-339a37d271c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:05.770988 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758aed94-f764-472c-ad14-339a37d271c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:05.782110 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758aed94-f764-472c-ad14-339a37d271c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:05.783547 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758aed94-f764-472c-ad14-339a37d271c5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:05.797822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-758aed94-f764-472c-ad14-339a37d271c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:05.797822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:05.797822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:05.797822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:05.797822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:05.797822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:05.797822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:05.797822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:05.797822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:05.797822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:05.797822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:05.797822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:05.797822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:05.797822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:05.797822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:05.797822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:05.798936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:05.798936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:05.798936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:05.798936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:05.798936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:05.798936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:05.798936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:05.798936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:05.798936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:05.798936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:05.798936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:05.798936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:05.798936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:05.798936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:05.798936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:05.798936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:05.798936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:05.800052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:05.800052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:05.800052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:05.800052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 277/577] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:05.800274 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758aed94-f764-472c-ad14-339a37d271c5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:05.811712 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c567a0-caad-48de-a4fb-43a47ce3234c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:05.812305 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c567a0-caad-48de-a4fb-43a47ce3234c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:05.812589 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c567a0-caad-48de-a4fb-43a47ce3234c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:05.812867 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c567a0-caad-48de-a4fb-43a47ce3234c None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:05.813636 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c567a0-caad-48de-a4fb-43a47ce3234c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:05.813945 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c567a0-caad-48de-a4fb-43a47ce3234c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:05.815373 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c567a0-caad-48de-a4fb-43a47ce3234c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:05.838404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 301/578] 10.222.0.18 () {56 vars in 1137 bytes} [Mon Aug 29 20:20:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:05.838404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c567a0-caad-48de-a4fb-43a47ce3234c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:05.911708 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758aed94-f764-472c-ad14-339a37d271c5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:05.932232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:05.940579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:05.952047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886141da-ee24-44bd-81b5-3bea1d04cedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:05.952767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886141da-ee24-44bd-81b5-3bea1d04cedf None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:05.953142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886141da-ee24-44bd-81b5-3bea1d04cedf None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:05.953510 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886141da-ee24-44bd-81b5-3bea1d04cedf None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:05.954269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c567a0-caad-48de-a4fb-43a47ce3234c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:05.954368 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886141da-ee24-44bd-81b5-3bea1d04cedf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:05.954780 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886141da-ee24-44bd-81b5-3bea1d04cedf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:05.956567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886141da-ee24-44bd-81b5-3bea1d04cedf None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:05.979640 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 278/579] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Aug 29 20:20:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:05.979640 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886141da-ee24-44bd-81b5-3bea1d04cedf None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:06.007216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e2e65-137e-4428-a327-020f2d66fd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:06.007846 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e2e65-137e-4428-a327-020f2d66fd77 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:06.008156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e2e65-137e-4428-a327-020f2d66fd77 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:06.008431 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e2e65-137e-4428-a327-020f2d66fd77 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:06.009117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e2e65-137e-4428-a327-020f2d66fd77 None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:06.009420 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e2e65-137e-4428-a327-020f2d66fd77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:06.010813 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e2e65-137e-4428-a327-020f2d66fd77 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:06.025580 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 302/580] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:20:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:06.025580 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e2e65-137e-4428-a327-020f2d66fd77 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:06.101183 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886141da-ee24-44bd-81b5-3bea1d04cedf None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:06.138405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e2e65-137e-4428-a327-020f2d66fd77 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:06.141100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b679df0-e9d3-4a6f-ae5b-86c46504adb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:06.141813 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b679df0-e9d3-4a6f-ae5b-86c46504adb2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:06.142181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b679df0-e9d3-4a6f-ae5b-86c46504adb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:06.142456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b679df0-e9d3-4a6f-ae5b-86c46504adb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:06.143092 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b679df0-e9d3-4a6f-ae5b-86c46504adb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:06.143388 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b679df0-e9d3-4a6f-ae5b-86c46504adb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:06.144788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b679df0-e9d3-4a6f-ae5b-86c46504adb2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:06.156806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 279/581] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:06.157461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b679df0-e9d3-4a6f-ae5b-86c46504adb2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:06.180915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073fd786-2aee-4987-ba32-6406c3097230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:06.181596 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fd786-2aee-4987-ba32-6406c3097230 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:06.181879 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fd786-2aee-4987-ba32-6406c3097230 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:06.182154 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fd786-2aee-4987-ba32-6406c3097230 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:06.194122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073fd786-2aee-4987-ba32-6406c3097230 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:06.195580 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073fd786-2aee-4987-ba32-6406c3097230 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:06.197555 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:06.208519 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:06.212057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-073fd786-2aee-4987-ba32-6406c3097230 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:06.212057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:06.212057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:06.212057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:06.212057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:06.212057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:06.212057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:06.212057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:06.212057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:06.212057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:06.212057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:06.212057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:06.212057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:06.212057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:06.212057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:06.212057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:06.213163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:06.213163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:06.213163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:06.213163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:06.213163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:06.213163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:06.213163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:06.213163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:06.213163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:06.213163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:06.213163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:06.213163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:06.213163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:06.213163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:06.213163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:06.213163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:06.213163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:06.214232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:06.214232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:06.214232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:06.220559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 303/582] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:06.220559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073fd786-2aee-4987-ba32-6406c3097230 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:06.281872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b679df0-e9d3-4a6f-ae5b-86c46504adb2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:06.326919 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e76b07-e5f6-4bf0-8e70-66401595b318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:06.327700 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e76b07-e5f6-4bf0-8e70-66401595b318 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:06.328070 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e76b07-e5f6-4bf0-8e70-66401595b318 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:06.328343 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e76b07-e5f6-4bf0-8e70-66401595b318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:06.338236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073fd786-2aee-4987-ba32-6406c3097230 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:06.339513 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e76b07-e5f6-4bf0-8e70-66401595b318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:06.341276 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e76b07-e5f6-4bf0-8e70-66401595b318 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:06.355499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-78e76b07-e5f6-4bf0-8e70-66401595b318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:06.355499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:06.355499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:06.355499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:06.355499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:06.355499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:06.355499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:06.355499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:06.355499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:06.355499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:06.355499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:06.355499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:06.355499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:06.355499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:06.355499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:06.355499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:06.357173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:06.357173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:06.357173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:06.357173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:06.357173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:06.357173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:06.357173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:06.357173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:06.357173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:06.357173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:06.357173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:06.357173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:06.357173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:06.357173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:06.357173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:06.357173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:06.357173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:06.362989 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:06.362989 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:06.362989 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:06.362989 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 280/583] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:06.362989 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e76b07-e5f6-4bf0-8e70-66401595b318 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:06.382969 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705357e1-bd20-41b7-b377-885dc1cd92ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:06.383694 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705357e1-bd20-41b7-b377-885dc1cd92ba None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:06.384096 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705357e1-bd20-41b7-b377-885dc1cd92ba None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:06.384448 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705357e1-bd20-41b7-b377-885dc1cd92ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:06.395548 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705357e1-bd20-41b7-b377-885dc1cd92ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:06.397094 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705357e1-bd20-41b7-b377-885dc1cd92ba None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:06.411586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-705357e1-bd20-41b7-b377-885dc1cd92ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:06.411586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:06.411586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:06.411586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:06.411586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:06.411586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:06.411586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:06.411586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:06.411586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:06.411586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:06.411586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:06.411586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:06.411586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:06.411586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:06.411586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:06.411586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:06.412744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:06.412744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:06.412744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:06.412744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:06.412744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:06.412744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:06.412744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:06.412744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:06.412744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:06.412744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:06.412744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:06.412744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:06.412744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:06.412744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:06.412744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:06.412744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:06.412744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:06.414098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:06.414098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:06.414098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:06.417680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 304/584] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:06.419147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705357e1-bd20-41b7-b377-885dc1cd92ba None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:06.461982 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:06.469188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e76b07-e5f6-4bf0-8e70-66401595b318 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:06.469712 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:06.506247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ba5fc-cbdd-4524-a72b-e4560a1d212b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:06.507050 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ba5fc-cbdd-4524-a72b-e4560a1d212b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:06.507411 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ba5fc-cbdd-4524-a72b-e4560a1d212b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:06.507778 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ba5fc-cbdd-4524-a72b-e4560a1d212b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:06.515579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ba5fc-cbdd-4524-a72b-e4560a1d212b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:06.517163 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ba5fc-cbdd-4524-a72b-e4560a1d212b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:06.528724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-448ba5fc-cbdd-4524-a72b-e4560a1d212b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 20:20:06.528724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:06.528724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:06.528724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:06.528724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:06.528724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:06.528724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:06.528724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:06.528724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:06.528724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:06.528724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:06.528724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:06.528724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:06.528724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:06.528724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:06.528724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:06.529852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:06.529852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:06.529852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:06.529852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:06.529852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:06.529852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:06.529852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:06.529852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:06.529852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:06.529852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:06.529852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:06.529852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:06.529852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:06.529852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:06.529852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:06.529852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:06.529852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:06.529852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:06.530972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:06.530972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:06.530972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:06.530972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:06.530972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:06.530972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:06.530972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:06.530972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:06.530972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 20:20:06.530972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:06.531610 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705357e1-bd20-41b7-b377-885dc1cd92ba None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:06.537042 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 281/585] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:20:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:06.537042 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ba5fc-cbdd-4524-a72b-e4560a1d212b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:06.573971 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546bb11d-ba46-4f14-8a2a-347eece1d007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:06.574622 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bb11d-ba46-4f14-8a2a-347eece1d007 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:06.574899 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bb11d-ba46-4f14-8a2a-347eece1d007 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:06.575174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546bb11d-ba46-4f14-8a2a-347eece1d007 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:06.575826 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546bb11d-ba46-4f14-8a2a-347eece1d007 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:06.576158 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546bb11d-ba46-4f14-8a2a-347eece1d007 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:06.577537 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546bb11d-ba46-4f14-8a2a-347eece1d007 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:06.594628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 305/586] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:06.594628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546bb11d-ba46-4f14-8a2a-347eece1d007 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:06.665843 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ba5fc-cbdd-4524-a72b-e4560a1d212b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:06.708909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798fdfe-4a9e-439f-9b11-56d184519d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:06.709522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798fdfe-4a9e-439f-9b11-56d184519d28 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:06.709798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798fdfe-4a9e-439f-9b11-56d184519d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:06.710065 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798fdfe-4a9e-439f-9b11-56d184519d28 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:06.710709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798fdfe-4a9e-439f-9b11-56d184519d28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:06.711008 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798fdfe-4a9e-439f-9b11-56d184519d28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:06.712412 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798fdfe-4a9e-439f-9b11-56d184519d28 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:06.725968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 282/587] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:06.727199 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798fdfe-4a9e-439f-9b11-56d184519d28 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:06.740907 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546bb11d-ba46-4f14-8a2a-347eece1d007 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:06.746730 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:06.749778 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:06.779609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e46a53-2e64-45c3-a7cb-0093e893aeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:06.780476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e46a53-2e64-45c3-a7cb-0093e893aeeb None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:06.780688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e46a53-2e64-45c3-a7cb-0093e893aeeb None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:06.780929 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e46a53-2e64-45c3-a7cb-0093e893aeeb None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:06.781583 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e46a53-2e64-45c3-a7cb-0093e893aeeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:06.781889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e46a53-2e64-45c3-a7cb-0093e893aeeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:06.783301 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e46a53-2e64-45c3-a7cb-0093e893aeeb None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:06.796523 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 306/588] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:06.797190 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e46a53-2e64-45c3-a7cb-0093e893aeeb None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:06.839665 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798fdfe-4a9e-439f-9b11-56d184519d28 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:06.878276 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4a693-cd8b-4361-9e72-6e5f38722d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:06.878885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4a693-cd8b-4361-9e72-6e5f38722d14 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:06.879166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4a693-cd8b-4361-9e72-6e5f38722d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:06.879436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4a693-cd8b-4361-9e72-6e5f38722d14 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:06.880191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a4a693-cd8b-4361-9e72-6e5f38722d14 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:06.880490 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a4a693-cd8b-4361-9e72-6e5f38722d14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:06.881912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a4a693-cd8b-4361-9e72-6e5f38722d14 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:06.900055 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 283/589] 10.222.0.18 () {56 vars in 1137 bytes} [Mon Aug 29 20:20:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:06.900898 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4a693-cd8b-4361-9e72-6e5f38722d14 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:06.922446 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e46a53-2e64-45c3-a7cb-0093e893aeeb None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:06.961124 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9fefb-e81d-4aff-b35a-b0a3cb95f381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:06.962007 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9fefb-e81d-4aff-b35a-b0a3cb95f381 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:06.962298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9fefb-e81d-4aff-b35a-b0a3cb95f381 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:06.963067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9fefb-e81d-4aff-b35a-b0a3cb95f381 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:06.972563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa9fefb-e81d-4aff-b35a-b0a3cb95f381 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:06.975130 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa9fefb-e81d-4aff-b35a-b0a3cb95f381 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:06.979667 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:06.989174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-1aa9fefb-e81d-4aff-b35a-b0a3cb95f381 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:06.989174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:06.989174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:06.989174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:06.989174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:06.989174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:06.989174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:06.989174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:06.989174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:06.989174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:06.989174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:06.989174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:06.989174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:06.989174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:06.989174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:06.990198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:06.990198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:06.990198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:06.990198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:06.990198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:06.990198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:06.990198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:06.990198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:06.990198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:06.990198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:06.990198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:06.990198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:06.990198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:06.990198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:06.990198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:06.990198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:06.990198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:06.991747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:06.991747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:06.991747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:06.991747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:06.991747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:06.991747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:06.991747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:06.991747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:06.991747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 307/590] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:06.991747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9fefb-e81d-4aff-b35a-b0a3cb95f381 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:07.014929 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4a693-cd8b-4361-9e72-6e5f38722d14 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:07.020251 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:07.052152 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33626316-78c8-4b34-bf2f-30eb7522ee7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:07.052778 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33626316-78c8-4b34-bf2f-30eb7522ee7b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:07.053055 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33626316-78c8-4b34-bf2f-30eb7522ee7b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:07.053498 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33626316-78c8-4b34-bf2f-30eb7522ee7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:07.060652 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33626316-78c8-4b34-bf2f-30eb7522ee7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:07.062081 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33626316-78c8-4b34-bf2f-30eb7522ee7b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:07.073223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-33626316-78c8-4b34-bf2f-30eb7522ee7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:07.073223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:07.073223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:07.073223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:07.073223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:07.073223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:07.073223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:07.073223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:07.073223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:07.073223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:07.073223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:07.073223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:07.073223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:07.073223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:07.073223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:07.074115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:07.074115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:07.074115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:07.074115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:07.074115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:07.074115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:07.074115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:07.074115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:07.074115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:07.074115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:07.074115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:07.074115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:07.074115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:07.074115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:07.074115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:07.074115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:07.074115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:07.075155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:07.075155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:07.075155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:07.075155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:07.075155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:07.075155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:07.075155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:07.075155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:07.075155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 284/591] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:07.075155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33626316-78c8-4b34-bf2f-30eb7522ee7b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:07.106271 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9fefb-e81d-4aff-b35a-b0a3cb95f381 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:07.140182 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146fbec-2f8e-4d73-afdf-b5657603ae9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:07.140812 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146fbec-2f8e-4d73-afdf-b5657603ae9c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:07.141089 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146fbec-2f8e-4d73-afdf-b5657603ae9c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:07.141362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146fbec-2f8e-4d73-afdf-b5657603ae9c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:07.148797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e146fbec-2f8e-4d73-afdf-b5657603ae9c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:07.150512 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e146fbec-2f8e-4d73-afdf-b5657603ae9c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:07.161914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-e146fbec-2f8e-4d73-afdf-b5657603ae9c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 20:20:07.161914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:07.161914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:07.161914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:07.161914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:07.161914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:07.161914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:07.161914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:07.161914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:07.161914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:07.161914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:07.161914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:07.161914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:07.161914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:07.161914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:07.161914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:07.162870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:07.162870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:07.162870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:07.162870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:07.162870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:07.162870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:07.162870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:07.162870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:07.162870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:07.162870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:07.162870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:07.162870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:07.162870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:07.162870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:07.162870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:07.162870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:07.162870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:07.162870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:07.164343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:07.164343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:07.164343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:07.164343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:07.164343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:07.164343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:07.164343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:07.164343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:07.164343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 20:20:07.164343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:07.164343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 308/592] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:20:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:07.164343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146fbec-2f8e-4d73-afdf-b5657603ae9c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:07.180683 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33626316-78c8-4b34-bf2f-30eb7522ee7b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:07.213144 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56aa71-b4d4-42b4-a9d3-1574d053d4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:07.213806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56aa71-b4d4-42b4-a9d3-1574d053d4ab None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:07.214080 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56aa71-b4d4-42b4-a9d3-1574d053d4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:07.214346 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56aa71-b4d4-42b4-a9d3-1574d053d4ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:07.216229 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:07.225811 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56aa71-b4d4-42b4-a9d3-1574d053d4ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:07.227260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56aa71-b4d4-42b4-a9d3-1574d053d4ab None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:07.242023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-9d56aa71-b4d4-42b4-a9d3-1574d053d4ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:07.242023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:07.242023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:07.242023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:07.242023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:07.242023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:07.242023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:07.242023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:07.242023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:07.242023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:07.242023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:07.242023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:07.242023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:07.242023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:07.242023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:07.242023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:07.242912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:07.242912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:07.242912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:07.242912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:07.242912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:07.242912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:07.242912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:07.242912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:07.242912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:07.242912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:07.242912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:07.242912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:07.242912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:07.242912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:07.242912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:07.242912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:07.242912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:07.243786 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:07.243786 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:07.243786 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:07.243786 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 285/593] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:07.244045 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56aa71-b4d4-42b4-a9d3-1574d053d4ab None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:07.260280 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:07.282321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146fbec-2f8e-4d73-afdf-b5657603ae9c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:07.317934 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68bc8a8-123f-4c5e-b191-e76ebf4b3689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:07.318547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bc8a8-123f-4c5e-b191-e76ebf4b3689 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:07.318832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bc8a8-123f-4c5e-b191-e76ebf4b3689 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:07.319107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bc8a8-123f-4c5e-b191-e76ebf4b3689 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:07.319801 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68bc8a8-123f-4c5e-b191-e76ebf4b3689 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:07.320132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68bc8a8-123f-4c5e-b191-e76ebf4b3689 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:07.321543 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68bc8a8-123f-4c5e-b191-e76ebf4b3689 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:07.333801 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 309/594] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:07.335032 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d5e62-3c2e-4827-b31d-cc58c9534a88 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:07.335321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d5e62-3c2e-4827-b31d-cc58c9534a88 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:07.335596 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d5e62-3c2e-4827-b31d-cc58c9534a88 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:07.340556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 310/595] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:07.340556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d5e62-3c2e-4827-b31d-cc58c9534a88 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:07.353265 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56aa71-b4d4-42b4-a9d3-1574d053d4ab None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:07.390236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a6ab4-3486-4dd2-874c-3257eda3d99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:07.390858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a6ab4-3486-4dd2-874c-3257eda3d99c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:07.391119 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a6ab4-3486-4dd2-874c-3257eda3d99c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:07.391502 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a6ab4-3486-4dd2-874c-3257eda3d99c None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:07.392222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a6ab4-3486-4dd2-874c-3257eda3d99c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:07.392517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a6ab4-3486-4dd2-874c-3257eda3d99c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:07.393948 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a6ab4-3486-4dd2-874c-3257eda3d99c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:07.406225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 286/596] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:07.407042 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a6ab4-3486-4dd2-874c-3257eda3d99c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:07.456140 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:07.469933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d5e62-3c2e-4827-b31d-cc58c9534a88 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:07.504755 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f03bfb-70a4-4a6d-b872-1a3139374619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:07.505362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f03bfb-70a4-4a6d-b872-1a3139374619 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:07.505643 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f03bfb-70a4-4a6d-b872-1a3139374619 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:07.505916 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f03bfb-70a4-4a6d-b872-1a3139374619 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:07.506643 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f03bfb-70a4-4a6d-b872-1a3139374619 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:07.506952 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f03bfb-70a4-4a6d-b872-1a3139374619 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:07.508400 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f03bfb-70a4-4a6d-b872-1a3139374619 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:07.509667 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a6ab4-3486-4dd2-874c-3257eda3d99c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:07.525004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 311/597] 10.222.0.18 () {56 vars in 1139 bytes} [Mon Aug 29 20:20:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:07.525798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f03bfb-70a4-4a6d-b872-1a3139374619 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:07.531857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:07.549721 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2eca38-af4f-4c9e-a6e3-f589f00e8a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:07.550322 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2eca38-af4f-4c9e-a6e3-f589f00e8a69 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:07.550596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2eca38-af4f-4c9e-a6e3-f589f00e8a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:07.550866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2eca38-af4f-4c9e-a6e3-f589f00e8a69 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:07.551496 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2eca38-af4f-4c9e-a6e3-f589f00e8a69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:07.551791 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2eca38-af4f-4c9e-a6e3-f589f00e8a69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:07.553174 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2eca38-af4f-4c9e-a6e3-f589f00e8a69 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:07.564321 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 287/598] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:07.565490 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7ad0b-f958-4687-af4d-287108709e84 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:07.565773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7ad0b-f958-4687-af4d-287108709e84 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:07.566041 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7ad0b-f958-4687-af4d-287108709e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:07.583915 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-53e7ad0b-f958-4687-af4d-287108709e84 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:07.585113 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-53e7ad0b-f958-4687-af4d-287108709e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:07.630232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f03bfb-70a4-4a6d-b872-1a3139374619 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:07.650280 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 288/599] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:20:07.651089 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7ad0b-f958-4687-af4d-287108709e84 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:07.669812 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db59ca8-1dce-4cff-a13a-94ea57cc9cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:07.670578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db59ca8-1dce-4cff-a13a-94ea57cc9cd4 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:07.670959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db59ca8-1dce-4cff-a13a-94ea57cc9cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:07.671325 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db59ca8-1dce-4cff-a13a-94ea57cc9cd4 None admin] PATH_INFO: `/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/85624cea63af40fda09f57ddf2cd5c3a/roles/04bc48cc5b6842029b93a845d6744985` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:07.689255 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:07.701317 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db59ca8-1dce-4cff-a13a-94ea57cc9cd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=90b25dc95d5a4a068994bdd801d6b8ea, user_id=85624cea63af40fda09f57ddf2cd5c3a, role_id=04bc48cc5b6842029b93a845d6744985)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:07.702781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db59ca8-1dce-4cff-a13a-94ea57cc9cd4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:07.728053 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 312/600] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:20:07 2022] PUT /identity/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/85624cea63af40fda09f57ddf2cd5c3a/roles/04bc48cc5b6842029b93a845d6744985 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:20:07.728053 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db59ca8-1dce-4cff-a13a-94ea57cc9cd4 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:07.776273 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7ad0b-f958-4687-af4d-287108709e84 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:07.809680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:07.817804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042b420-76ca-44e8-8a97-4dbee143a497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:07.818557 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042b420-76ca-44e8-8a97-4dbee143a497 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:07.818832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042b420-76ca-44e8-8a97-4dbee143a497 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:07.819102 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042b420-76ca-44e8-8a97-4dbee143a497 None admin] PATH_INFO: `/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/c0c44754a1c04a2b87081aba6cb509d3/roles/04bc48cc5b6842029b93a845d6744985` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:07.835611 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042b420-76ca-44e8-8a97-4dbee143a497 None admin] RBAC: Authorizing `identity:create_grant(project_id=90b25dc95d5a4a068994bdd801d6b8ea, user_id=c0c44754a1c04a2b87081aba6cb509d3, role_id=04bc48cc5b6842029b93a845d6744985)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:07.837037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042b420-76ca-44e8-8a97-4dbee143a497 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:07.860615 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 289/601] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:20:07 2022] PUT /identity/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/c0c44754a1c04a2b87081aba6cb509d3/roles/04bc48cc5b6842029b93a845d6744985 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:20:07.861809 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042b420-76ca-44e8-8a97-4dbee143a497 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:07.878604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db59ca8-1dce-4cff-a13a-94ea57cc9cd4 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:07.927927 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031fe77-64bc-4215-a36c-6dff5090d767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:07.928610 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031fe77-64bc-4215-a36c-6dff5090d767 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:07.928896 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031fe77-64bc-4215-a36c-6dff5090d767 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:07.929172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031fe77-64bc-4215-a36c-6dff5090d767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:07.943134 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031fe77-64bc-4215-a36c-6dff5090d767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:07.945784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031fe77-64bc-4215-a36c-6dff5090d767 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:07.964615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-4031fe77-64bc-4215-a36c-6dff5090d767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:07.964615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:07.964615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:07.964615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:07.964615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:07.964615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:07.964615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:07.964615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:07.964615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:07.964615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:07.964615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:07.964615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:07.964615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:07.964615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:07.964615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:07.964615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:07.967624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:07.967624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:07.967624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:07.967624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:07.967624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:07.967624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:07.967624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:07.967624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:07.967624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:07.967624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:07.967624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:07.967624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:07.967624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:07.967624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:07.967624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:07.967624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:07.967624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:07.970531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:07.970531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:07.970531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:07.970531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 313/602] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:07.970531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031fe77-64bc-4215-a36c-6dff5090d767 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:08.018560 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:08.042020 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042b420-76ca-44e8-8a97-4dbee143a497 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:08.089442 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6facfe68-775f-4d14-a820-388990f60b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:08.090096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6facfe68-775f-4d14-a820-388990f60b0b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:08.091639 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6facfe68-775f-4d14-a820-388990f60b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:08.091912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6facfe68-775f-4d14-a820-388990f60b0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:08.105261 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6facfe68-775f-4d14-a820-388990f60b0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:08.107657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6facfe68-775f-4d14-a820-388990f60b0b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:08.114823 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031fe77-64bc-4215-a36c-6dff5090d767 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:08.124606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-6facfe68-775f-4d14-a820-388990f60b0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:08.124606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:08.124606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:08.124606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:08.124606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:08.124606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:08.124606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:08.124606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:08.124606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:08.124606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:08.124606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:08.124606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:08.124606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:08.124606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:08.124606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:08.125507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:08.125507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:08.125507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:08.125507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:08.125507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:08.125507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:08.125507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:08.125507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:08.125507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:08.125507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:08.125507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:08.125507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:08.125507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:08.125507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:08.125507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:08.125507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:08.125507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:08.126391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:08.126391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:08.126391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:08.126391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:08.126391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:08.126391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:08.126391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:08.126391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:08.129015 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 290/603] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:08.129410 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:08.134614 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6facfe68-775f-4d14-a820-388990f60b0b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:08.171232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded68a17-1a3c-4a02-8415-aaa17a426126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:08.172172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded68a17-1a3c-4a02-8415-aaa17a426126 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:08.172458 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded68a17-1a3c-4a02-8415-aaa17a426126 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:08.172738 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded68a17-1a3c-4a02-8415-aaa17a426126 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:08.187281 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded68a17-1a3c-4a02-8415-aaa17a426126 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:08.194654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-ded68a17-1a3c-4a02-8415-aaa17a426126 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:08.194654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:08.194654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:08.194654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:08.194654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:08.194654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:08.194654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:08.194654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:08.194654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:08.194654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:08.194654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:08.194654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:08.194654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:08.194654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:08.194654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:08.194654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:08.195599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:08.195599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:08.195599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:08.195599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:08.195599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:08.195599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:08.195599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:08.195599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:08.195599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:08.195599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:08.195599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:08.195599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:08.195599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:08.195599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:08.195599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:08.195599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:08.196450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:08.196450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:08.196450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:08.196450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:08.196450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:08.196450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:08.196450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:08.196450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:08.196450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:08.196450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:08.196450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:08.196450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:08.200013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 314/604] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:20:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:08.201115 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded68a17-1a3c-4a02-8415-aaa17a426126 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:08.254577 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6facfe68-775f-4d14-a820-388990f60b0b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:08.308743 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662dd958-caaa-4947-a287-f1462abe02c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:08.310134 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662dd958-caaa-4947-a287-f1462abe02c5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:08.310615 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662dd958-caaa-4947-a287-f1462abe02c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:08.311102 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662dd958-caaa-4947-a287-f1462abe02c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:08.312434 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662dd958-caaa-4947-a287-f1462abe02c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:08.313340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662dd958-caaa-4947-a287-f1462abe02c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:08.315214 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662dd958-caaa-4947-a287-f1462abe02c5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:08.332373 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 291/605] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:08.333630 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662dd958-caaa-4947-a287-f1462abe02c5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:08.340570 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded68a17-1a3c-4a02-8415-aaa17a426126 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:08.346271 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:08.384723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8aa457-4223-431c-9a45-625032212993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:08.386683 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8aa457-4223-431c-9a45-625032212993 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:08.387722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8aa457-4223-431c-9a45-625032212993 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:08.388221 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8aa457-4223-431c-9a45-625032212993 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:08.389121 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8aa457-4223-431c-9a45-625032212993 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:08.389635 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8aa457-4223-431c-9a45-625032212993 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:08.391253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8aa457-4223-431c-9a45-625032212993 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:08.408351 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 315/606] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:08.410100 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8aa457-4223-431c-9a45-625032212993 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:08.446236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:08.460278 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662dd958-caaa-4947-a287-f1462abe02c5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:08.498241 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc82b08-4a84-49ce-941d-ee1645f62b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:08.498863 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc82b08-4a84-49ce-941d-ee1645f62b3b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:08.499152 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc82b08-4a84-49ce-941d-ee1645f62b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:08.499464 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc82b08-4a84-49ce-941d-ee1645f62b3b None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:08.500254 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc82b08-4a84-49ce-941d-ee1645f62b3b None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:08.500558 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc82b08-4a84-49ce-941d-ee1645f62b3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:08.501957 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc82b08-4a84-49ce-941d-ee1645f62b3b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:08.516204 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 292/607] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:20:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:08.516204 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc82b08-4a84-49ce-941d-ee1645f62b3b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:08.541857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8aa457-4223-431c-9a45-625032212993 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:08.580535 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe95902-4e52-4948-9fc5-30291db7c43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:08.581243 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe95902-4e52-4948-9fc5-30291db7c43e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:08.581620 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe95902-4e52-4948-9fc5-30291db7c43e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:08.582078 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe95902-4e52-4948-9fc5-30291db7c43e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:08.588963 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe95902-4e52-4948-9fc5-30291db7c43e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:08.591110 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe95902-4e52-4948-9fc5-30291db7c43e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:08.592329 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:08.604760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-dfe95902-4e52-4948-9fc5-30291db7c43e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:08.604760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:08.604760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:08.604760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:08.604760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:08.604760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:08.604760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:08.604760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:08.604760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:08.604760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:08.604760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:08.604760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:08.604760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:08.604760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:08.604760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:08.606299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:08.606299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:08.606299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:08.606299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:08.606299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:08.606299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:08.606299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:08.606299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:08.606299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:08.606299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:08.606299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:08.606299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:08.606299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:08.606299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:08.606299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:08.606299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:08.606299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:08.607309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:08.607309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:08.607309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:08.607309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:08.607309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:08.607309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:08.607309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:08.607309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:08.612686 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 316/608] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:08.613820 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe95902-4e52-4948-9fc5-30291db7c43e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:08.638161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc82b08-4a84-49ce-941d-ee1645f62b3b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:08.677568 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12e818-a093-42f8-992c-9279c19c5b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:08.678550 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12e818-a093-42f8-992c-9279c19c5b27 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:08.679073 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12e818-a093-42f8-992c-9279c19c5b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:08.679582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12e818-a093-42f8-992c-9279c19c5b27 None admin] PATH_INFO: `/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/5d022ed8d2ac43ae816aa396a8e2f17a/roles/04bc48cc5b6842029b93a845d6744985` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:08.696568 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12e818-a093-42f8-992c-9279c19c5b27 None admin] RBAC: Authorizing `identity:create_grant(project_id=90b25dc95d5a4a068994bdd801d6b8ea, user_id=5d022ed8d2ac43ae816aa396a8e2f17a, role_id=04bc48cc5b6842029b93a845d6744985)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:08.698591 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12e818-a093-42f8-992c-9279c19c5b27 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:08.717214 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 293/609] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:20:08 2022] PUT /identity/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/5d022ed8d2ac43ae816aa396a8e2f17a/roles/04bc48cc5b6842029b93a845d6744985 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:20:08.718487 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12e818-a093-42f8-992c-9279c19c5b27 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:08.728101 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:08.747514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe95902-4e52-4948-9fc5-30291db7c43e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:08.807861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3dd7c-69db-4e3c-a11a-a1ccbbe53c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:08.807861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3dd7c-69db-4e3c-a11a-a1ccbbe53c19 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:08.808201 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3dd7c-69db-4e3c-a11a-a1ccbbe53c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:08.808556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c3dd7c-69db-4e3c-a11a-a1ccbbe53c19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:08.833385 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3dd7c-69db-4e3c-a11a-a1ccbbe53c19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:08.834769 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c3dd7c-69db-4e3c-a11a-a1ccbbe53c19 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:08.846493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12e818-a093-42f8-992c-9279c19c5b27 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:08.862035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-24c3dd7c-69db-4e3c-a11a-a1ccbbe53c19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:08.862035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:08.862035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:08.862035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:08.862035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:08.862035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:08.862035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:08.862035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:08.862035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:08.862035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:08.862035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:08.862035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:08.862035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:08.862035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:08.862035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:08.862035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:08.863060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:08.863060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:08.863060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:08.863060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:08.863060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:08.863060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:08.863060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:08.863060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:08.863060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:08.863060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:08.863060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:08.863060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:08.863060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:08.863060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:08.863060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:08.863060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:08.863060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:08.864047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:08.864047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:08.864047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:08.868696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 317/610] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:08.870249 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3dd7c-69db-4e3c-a11a-a1ccbbe53c19 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:08.871061 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:08.896165 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603502ed-6340-460c-86f4-477ce0fe676a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:08.897291 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603502ed-6340-460c-86f4-477ce0fe676a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:08.897792 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603502ed-6340-460c-86f4-477ce0fe676a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:08.898289 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603502ed-6340-460c-86f4-477ce0fe676a None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:08.899383 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603502ed-6340-460c-86f4-477ce0fe676a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:08.900010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603502ed-6340-460c-86f4-477ce0fe676a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:08.901758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603502ed-6340-460c-86f4-477ce0fe676a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:08.916351 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 294/611] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:08.922800 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603502ed-6340-460c-86f4-477ce0fe676a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:09.062835 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c3dd7c-69db-4e3c-a11a-a1ccbbe53c19 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:09.070740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603502ed-6340-460c-86f4-477ce0fe676a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:09.120491 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10136d48-50d4-4f4f-af71-d1af7c0bf9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:09.121077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10136d48-50d4-4f4f-af71-d1af7c0bf9a2 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:09.121953 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10136d48-50d4-4f4f-af71-d1af7c0bf9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:09.122233 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10136d48-50d4-4f4f-af71-d1af7c0bf9a2 None admin] PATH_INFO: `/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/d51e331a4c9c4f038bdb80f8c5b74e42/roles/04bc48cc5b6842029b93a845d6744985` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:09.131904 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168eeab-9f3d-43ed-ab15-303d8b693aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:09.132573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168eeab-9f3d-43ed-ab15-303d8b693aad None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:09.132863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168eeab-9f3d-43ed-ab15-303d8b693aad None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:09.133143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168eeab-9f3d-43ed-ab15-303d8b693aad None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:09.133812 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168eeab-9f3d-43ed-ab15-303d8b693aad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:09.134122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168eeab-9f3d-43ed-ab15-303d8b693aad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:09.135501 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168eeab-9f3d-43ed-ab15-303d8b693aad None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:09.140809 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10136d48-50d4-4f4f-af71-d1af7c0bf9a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=90b25dc95d5a4a068994bdd801d6b8ea, user_id=d51e331a4c9c4f038bdb80f8c5b74e42, role_id=04bc48cc5b6842029b93a845d6744985)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:09.142242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10136d48-50d4-4f4f-af71-d1af7c0bf9a2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:09.152278 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:09.162612 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 318/612] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:09.169981 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168eeab-9f3d-43ed-ab15-303d8b693aad None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:09.174306 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 295/613] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:20:08 2022] PUT /identity/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/d51e331a4c9c4f038bdb80f8c5b74e42/roles/04bc48cc5b6842029b93a845d6744985 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:20:09.287093 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168eeab-9f3d-43ed-ab15-303d8b693aad None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:09.325471 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2c884-0d6c-446d-8770-479499247db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:09.326114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2c884-0d6c-446d-8770-479499247db2 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:09.326393 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2c884-0d6c-446d-8770-479499247db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:09.326754 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2c884-0d6c-446d-8770-479499247db2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:09.335036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2c884-0d6c-446d-8770-479499247db2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:09.336667 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2c884-0d6c-446d-8770-479499247db2 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:09.350646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-9dd2c884-0d6c-446d-8770-479499247db2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:20:09.350646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:09.350646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:09.350646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:09.350646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:09.350646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:09.350646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:09.350646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:09.350646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:09.350646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:09.350646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:09.350646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:09.350646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:09.350646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:09.350646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:09.350646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:09.352117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:09.352117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:09.352117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:09.352117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:09.352117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:09.352117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:09.352117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:09.352117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:09.352117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:09.352117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:09.352117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:09.352117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:09.352117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:09.352117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:09.352117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:09.352117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:09.352117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:09.352117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:09.353036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:09.353036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:09.353036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:09.353036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:09.353036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:09.353036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:09.353036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:09.353036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:09.353036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:20:09.353036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:09.356068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 319/614] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:20:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:09.358219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10136d48-50d4-4f4f-af71-d1af7c0bf9a2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:09.399699 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:09.482221 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10136d48-50d4-4f4f-af71-d1af7c0bf9a2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:09.523481 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc30745-51be-4b9d-a814-0ad45c677f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:09.524144 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc30745-51be-4b9d-a814-0ad45c677f9a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:09.524419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc30745-51be-4b9d-a814-0ad45c677f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:09.524687 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc30745-51be-4b9d-a814-0ad45c677f9a None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:09.525475 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc30745-51be-4b9d-a814-0ad45c677f9a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:09.525781 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc30745-51be-4b9d-a814-0ad45c677f9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:09.527237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc30745-51be-4b9d-a814-0ad45c677f9a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:09.553186 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 296/615] 10.222.0.18 () {56 vars in 1135 bytes} [Mon Aug 29 20:20:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:09.554674 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2c884-0d6c-446d-8770-479499247db2 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:09.672409 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:09.728191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2c884-0d6c-446d-8770-479499247db2 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:09.767229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe16bb-edd1-408c-be90-1be654595325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:09.767869 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe16bb-edd1-408c-be90-1be654595325 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:09.768174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe16bb-edd1-408c-be90-1be654595325 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:09.768447 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe16bb-edd1-408c-be90-1be654595325 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:09.780735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe16bb-edd1-408c-be90-1be654595325 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:09.782418 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe16bb-edd1-408c-be90-1be654595325 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:09.798095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-e1fe16bb-edd1-408c-be90-1be654595325 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:09.798095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:09.798095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:09.798095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:09.798095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:09.798095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:09.798095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:09.798095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:09.798095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:09.798095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:09.798095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:09.798095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:09.798095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:09.798095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:09.798095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:09.798095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:09.799114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:09.799114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:09.799114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:09.799114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:09.799114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:09.799114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:09.799114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:09.799114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:09.799114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:09.799114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:09.799114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:09.799114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:09.799114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:09.799114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:09.799114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:09.799114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:09.799114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:09.800076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:09.800076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:09.800076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:09.803947 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 320/616] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:09.805367 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc30745-51be-4b9d-a814-0ad45c677f9a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:09.907296 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:09.928796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc30745-51be-4b9d-a814-0ad45c677f9a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:09.974808 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d677e-b231-4b3e-9b66-de7b4af788c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:09.975723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d677e-b231-4b3e-9b66-de7b4af788c3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:09.976049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d677e-b231-4b3e-9b66-de7b4af788c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:09.976324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d677e-b231-4b3e-9b66-de7b4af788c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:09.976970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d677e-b231-4b3e-9b66-de7b4af788c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:09.977268 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d677e-b231-4b3e-9b66-de7b4af788c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:09.978629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d677e-b231-4b3e-9b66-de7b4af788c3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:09.999052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 297/617] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:10.000480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe16bb-edd1-408c-be90-1be654595325 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:10.009051 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5eac3-da7a-42fc-ab1b-4f0e00ab8903 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:10.009317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5eac3-da7a-42fc-ab1b-4f0e00ab8903 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:10.009603 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5eac3-da7a-42fc-ab1b-4f0e00ab8903 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:10.011212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 298/618] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:10.016306 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e01b25-3a5f-418a-bd3f-c8677302ff35 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:10.016590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e01b25-3a5f-418a-bd3f-c8677302ff35 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:10.016869 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e01b25-3a5f-418a-bd3f-c8677302ff35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:10.048293 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-43e01b25-3a5f-418a-bd3f-c8677302ff35 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:10.049515 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-43e01b25-3a5f-418a-bd3f-c8677302ff35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:10.122211 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 299/619] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:20:10.123512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89360e-ade5-4a7e-8adc-81b601d45396 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:10.123796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89360e-ade5-4a7e-8adc-81b601d45396 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:10.130701 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89360e-ade5-4a7e-8adc-81b601d45396 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:10.132453 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 300/620] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:10 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:10.146693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d23f57-e101-4887-8411-3e3a9f04c343 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:10.146982 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d23f57-e101-4887-8411-3e3a9f04c343 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:10.147255 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d23f57-e101-4887-8411-3e3a9f04c343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:10.160508 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe16bb-edd1-408c-be90-1be654595325 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:10.193068 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-a1d23f57-e101-4887-8411-3e3a9f04c343 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:10.193748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:10.194222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a1d23f57-e101-4887-8411-3e3a9f04c343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:10.203355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be4a33-5005-4d26-a8e4-a93b2a81cf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:10.204050 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be4a33-5005-4d26-a8e4-a93b2a81cf83 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:10.204341 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be4a33-5005-4d26-a8e4-a93b2a81cf83 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:10.204621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be4a33-5005-4d26-a8e4-a93b2a81cf83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:10.212744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be4a33-5005-4d26-a8e4-a93b2a81cf83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:10.214224 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be4a33-5005-4d26-a8e4-a93b2a81cf83 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:10.228279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-d3be4a33-5005-4d26-a8e4-a93b2a81cf83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:10.228279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:10.228279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:10.228279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:10.228279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:10.228279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:10.228279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:10.228279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:10.228279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:10.228279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:10.228279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:10.228279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:10.228279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:10.228279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:10.228279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:10.229229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:10.229229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:10.229229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:10.229229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:10.229229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:10.229229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:10.229229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:10.229229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:10.229229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:10.229229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:10.229229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:10.229229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:10.229229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:10.229229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:10.229229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:10.229229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:10.229229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:10.230106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:10.230106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:10.230106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:10.230106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:10.230106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:10.230106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:10.230106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:10.230106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:10.233642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 321/621] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:10.236416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be4a33-5005-4d26-a8e4-a93b2a81cf83 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:10.353335 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be4a33-5005-4d26-a8e4-a93b2a81cf83 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:10.390439 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fbc8f-0605-4b7f-a7b8-81d6f6feb828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:10.391055 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fbc8f-0605-4b7f-a7b8-81d6f6feb828 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:10.391338 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fbc8f-0605-4b7f-a7b8-81d6f6feb828 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:10.391611 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fbc8f-0605-4b7f-a7b8-81d6f6feb828 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:10.392405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fbc8f-0605-4b7f-a7b8-81d6f6feb828 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:10.393152 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fbc8f-0605-4b7f-a7b8-81d6f6feb828 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:10.394588 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fbc8f-0605-4b7f-a7b8-81d6f6feb828 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:10.409612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 301/623] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:20:10.412805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 322/623] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:10.414427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d23f57-e101-4887-8411-3e3a9f04c343 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:10.427441 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:10.515311 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fbc8f-0605-4b7f-a7b8-81d6f6feb828 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:10.572302 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:10.595158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d23f57-e101-4887-8411-3e3a9f04c343 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:10.669068 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8b72b-a1cf-41e4-8e28-ff0c95959737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:10.669752 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8b72b-a1cf-41e4-8e28-ff0c95959737 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:10.670033 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8b72b-a1cf-41e4-8e28-ff0c95959737 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:10.670307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8b72b-a1cf-41e4-8e28-ff0c95959737 None admin] PATH_INFO: `/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/599129b7d8ff47efb145787414ddb80d/roles/04bc48cc5b6842029b93a845d6744985` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:10.702272 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8b72b-a1cf-41e4-8e28-ff0c95959737 None admin] RBAC: Authorizing `identity:create_grant(project_id=90b25dc95d5a4a068994bdd801d6b8ea, user_id=599129b7d8ff47efb145787414ddb80d, role_id=04bc48cc5b6842029b93a845d6744985)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:10.703770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8b72b-a1cf-41e4-8e28-ff0c95959737 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:10.711814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fbc8f-0605-4b7f-a7b8-81d6f6feb828 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:10.729192 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 302/624] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:20:10 2022] PUT /identity/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/599129b7d8ff47efb145787414ddb80d/roles/04bc48cc5b6842029b93a845d6744985 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:20:10.763963 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b320ec3-1c84-4681-9da5-cce7cea73b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:10.764918 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b320ec3-1c84-4681-9da5-cce7cea73b5f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:10.765441 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b320ec3-1c84-4681-9da5-cce7cea73b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:10.765833 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b320ec3-1c84-4681-9da5-cce7cea73b5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:10.783532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b320ec3-1c84-4681-9da5-cce7cea73b5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:10.789060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-1b320ec3-1c84-4681-9da5-cce7cea73b5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:10.789060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:10.789060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:10.789060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:10.789060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:10.789060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:10.789060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:10.789060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:10.789060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:10.789060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:10.789060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:10.789060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:10.789060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:10.789060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:10.789060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:10.789060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:10.790040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:10.790040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:10.790040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:10.790040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:10.790040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:10.790040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:10.790040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:10.790040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:10.790040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:10.790040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:10.790040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:10.790040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:10.790040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:10.790040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:10.790040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:10.790040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:10.790864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:10.790864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:10.790864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:10.790864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:10.790864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:10.790864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:10.790864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:10.790864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:10.790864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:10.790864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:10.790864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:10.790864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:10.791517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8b72b-a1cf-41e4-8e28-ff0c95959737 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:10.791591 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 323/625] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:20:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:10.798274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b320ec3-1c84-4681-9da5-cce7cea73b5f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:10.881531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:10.937096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8b72b-a1cf-41e4-8e28-ff0c95959737 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:10.977354 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f96696-efab-416f-b95e-29460fe29b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:10.978002 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96696-efab-416f-b95e-29460fe29b55 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:10.978281 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96696-efab-416f-b95e-29460fe29b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:10.978550 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96696-efab-416f-b95e-29460fe29b55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:10.990432 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f96696-efab-416f-b95e-29460fe29b55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:10.990962 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b320ec3-1c84-4681-9da5-cce7cea73b5f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:10.995297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-25f96696-efab-416f-b95e-29460fe29b55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:10.995297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:10.995297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:10.995297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:10.995297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:10.995297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:10.995297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:10.995297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:10.995297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:10.995297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:10.995297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:10.995297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:10.995297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:10.995297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:10.995297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:10.995297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:10.996806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:10.996806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:10.996806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:10.996806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:10.996806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:10.996806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:10.996806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:10.996806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:10.996806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:10.996806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:10.996806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:10.996806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:10.996806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:10.996806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:10.996806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:10.996806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:10.997619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:10.997619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:10.997619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:10.997619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:10.997619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:10.997619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:10.997619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:10.997619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:10.997619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:10.997619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:10.997619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:10.997619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:10.997619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 303/626] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:20:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:10.999569 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:11.007000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f96696-efab-416f-b95e-29460fe29b55 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:11.056945 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee5163-c324-402c-8a2f-a0887151771b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:11.057625 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee5163-c324-402c-8a2f-a0887151771b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:11.057915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee5163-c324-402c-8a2f-a0887151771b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:11.058197 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee5163-c324-402c-8a2f-a0887151771b None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:11.059531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee5163-c324-402c-8a2f-a0887151771b None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:11.059965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee5163-c324-402c-8a2f-a0887151771b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:11.061427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee5163-c324-402c-8a2f-a0887151771b None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:11.081708 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 324/627] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:20:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:11.087223 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee5163-c324-402c-8a2f-a0887151771b None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:11.122405 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f96696-efab-416f-b95e-29460fe29b55 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:11.148270 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:11.189677 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed363e-33d3-4385-a10b-0675fc4464fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:11.190306 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed363e-33d3-4385-a10b-0675fc4464fd None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:11.190586 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed363e-33d3-4385-a10b-0675fc4464fd None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:11.190858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed363e-33d3-4385-a10b-0675fc4464fd None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:11.191535 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed363e-33d3-4385-a10b-0675fc4464fd None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:11.191835 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed363e-33d3-4385-a10b-0675fc4464fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:11.193379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed363e-33d3-4385-a10b-0675fc4464fd None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:11.203228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 304/628] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:20:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:11.210126 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed363e-33d3-4385-a10b-0675fc4464fd None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:11.252230 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee5163-c324-402c-8a2f-a0887151771b None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:11.313921 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653c596-315f-4515-8433-2ee15ad39c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:11.314892 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653c596-315f-4515-8433-2ee15ad39c6a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:11.315377 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653c596-315f-4515-8433-2ee15ad39c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:11.315852 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653c596-315f-4515-8433-2ee15ad39c6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:11.327303 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653c596-315f-4515-8433-2ee15ad39c6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:11.329134 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653c596-315f-4515-8433-2ee15ad39c6a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:11.344819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-6653c596-315f-4515-8433-2ee15ad39c6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:11.344819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:11.344819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:11.344819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:11.344819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:11.344819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:11.344819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:11.344819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:11.344819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:11.344819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:11.344819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:11.344819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:11.344819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:11.344819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:11.344819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:11.344819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:11.345798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:11.345798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:11.345798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:11.345798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:11.345798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:11.345798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:11.345798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:11.345798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:11.345798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:11.345798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:11.345798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:11.345798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:11.345798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:11.345798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:11.345798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:11.345798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:11.345798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:11.346666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:11.346666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:11.346666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:11.348745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 325/629] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:11.354564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e5c45-d025-4c66-906c-fee4f55ee61f None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:11.356284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e5c45-d025-4c66-906c-fee4f55ee61f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:11.356783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e5c45-d025-4c66-906c-fee4f55ee61f None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:11.358601 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 326/630] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:11.359542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e5c45-d025-4c66-906c-fee4f55ee61f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:11.364178 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:11.389277 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed363e-33d3-4385-a10b-0675fc4464fd None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:11.426632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7432fa2e-f84b-441c-bff4-5c40f3394b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:11.427300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432fa2e-f84b-441c-bff4-5c40f3394b84 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:11.427595 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432fa2e-f84b-441c-bff4-5c40f3394b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:11.427896 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432fa2e-f84b-441c-bff4-5c40f3394b84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:11.440152 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7432fa2e-f84b-441c-bff4-5c40f3394b84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:11.441626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7432fa2e-f84b-441c-bff4-5c40f3394b84 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:11.460834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-7432fa2e-f84b-441c-bff4-5c40f3394b84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:11.460834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:11.460834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:11.460834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:11.460834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:11.460834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:11.460834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:11.460834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:11.460834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:11.460834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:11.460834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:11.460834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:11.460834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:11.460834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:11.460834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:11.460834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:11.461822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:11.461822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:11.461822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:11.461822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:11.461822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:11.461822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:11.461822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:11.461822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:11.461822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:11.461822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:11.461822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:11.461822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:11.461822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:11.461822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:11.461822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:11.461822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:11.461822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:11.462687 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:11.462687 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:11.462687 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:11.462687 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 305/631] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:11.464012 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489ff8e-b704-48d9-b69b-cc2ba7235047 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:11.464277 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489ff8e-b704-48d9-b69b-cc2ba7235047 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:11.464716 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489ff8e-b704-48d9-b69b-cc2ba7235047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:11.474851 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e5c45-d025-4c66-906c-fee4f55ee61f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:11.486141 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e489ff8e-b704-48d9-b69b-cc2ba7235047 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:11.487676 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e489ff8e-b704-48d9-b69b-cc2ba7235047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:11.512910 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed097e6-e54e-4e6d-a536-b22aeec33534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:11.513889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed097e6-e54e-4e6d-a536-b22aeec33534 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:11.514380 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed097e6-e54e-4e6d-a536-b22aeec33534 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:11.514900 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed097e6-e54e-4e6d-a536-b22aeec33534 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:11.515589 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:11.517002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed097e6-e54e-4e6d-a536-b22aeec33534 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:11.517592 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed097e6-e54e-4e6d-a536-b22aeec33534 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:11.519462 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed097e6-e54e-4e6d-a536-b22aeec33534 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:11.537080 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 327/632] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:11.538370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed097e6-e54e-4e6d-a536-b22aeec33534 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:11.571491 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 306/633] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:20:11.572411 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489ff8e-b704-48d9-b69b-cc2ba7235047 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:11.644374 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:11.693702 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed097e6-e54e-4e6d-a536-b22aeec33534 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:11.740216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0bf9b-6802-481d-8644-1983cdc818f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:11.740838 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0bf9b-6802-481d-8644-1983cdc818f0 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:11.741117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0bf9b-6802-481d-8644-1983cdc818f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:11.741412 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0bf9b-6802-481d-8644-1983cdc818f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:11.742343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0bf9b-6802-481d-8644-1983cdc818f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:11.742662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0bf9b-6802-481d-8644-1983cdc818f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:11.744088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0bf9b-6802-481d-8644-1983cdc818f0 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:11.754311 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489ff8e-b704-48d9-b69b-cc2ba7235047 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:11.757311 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 328/634] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:11.758527 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f15b83-37c7-45aa-b939-b6cef18887a4 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:11.758822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f15b83-37c7-45aa-b939-b6cef18887a4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:11.759099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f15b83-37c7-45aa-b939-b6cef18887a4 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:11.760629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 329/635] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:11.768086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4eb16-43f4-49a9-aa00-c39e3c1cdb2a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:11.768367 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4eb16-43f4-49a9-aa00-c39e3c1cdb2a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:11.768673 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4eb16-43f4-49a9-aa00-c39e3c1cdb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:11.788185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-5cf4eb16-43f4-49a9-aa00-c39e3c1cdb2a None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:11.789716 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5cf4eb16-43f4-49a9-aa00-c39e3c1cdb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:11.797826 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852496a9-d0b5-4f07-b8cc-8cfc0e319d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:11.798455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852496a9-d0b5-4f07-b8cc-8cfc0e319d48 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:11.798732 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852496a9-d0b5-4f07-b8cc-8cfc0e319d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:11.799000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852496a9-d0b5-4f07-b8cc-8cfc0e319d48 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:11.806351 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852496a9-d0b5-4f07-b8cc-8cfc0e319d48 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:11.807760 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852496a9-d0b5-4f07-b8cc-8cfc0e319d48 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:11.819909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-852496a9-d0b5-4f07-b8cc-8cfc0e319d48 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:20:11.819909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:11.819909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:11.819909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:11.819909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:11.819909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:11.819909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:11.819909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:11.819909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:11.819909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:11.819909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:11.819909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:11.819909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:11.819909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:11.819909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:11.819909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:11.820924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:11.820924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:11.820924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:11.820924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:11.820924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:11.820924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:11.820924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:11.820924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:11.820924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:11.820924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:11.820924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:11.820924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:11.820924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:11.820924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:11.820924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:11.820924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:11.820924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:11.820924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:11.821922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:11.821922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:11.821922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:11.821922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:11.821922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:11.821922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:11.821922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:11.821922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:11.821922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:20:11.821922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:11.821922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 307/636] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:20:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:11.822543 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852496a9-d0b5-4f07-b8cc-8cfc0e319d48 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:11.858169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:11.884387 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 330/637] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:20:11.884987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4eb16-43f4-49a9-aa00-c39e3c1cdb2a None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:11.975094 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:11.978334 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852496a9-d0b5-4f07-b8cc-8cfc0e319d48 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:12.037201 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413efcc2-770c-4ff0-9d11-a36d393b65e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:12.037836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413efcc2-770c-4ff0-9d11-a36d393b65e8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:12.038119 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413efcc2-770c-4ff0-9d11-a36d393b65e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:12.038393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413efcc2-770c-4ff0-9d11-a36d393b65e8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:12.045057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4eb16-43f4-49a9-aa00-c39e3c1cdb2a None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:12.053107 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413efcc2-770c-4ff0-9d11-a36d393b65e8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:12.054994 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413efcc2-770c-4ff0-9d11-a36d393b65e8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:12.068747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-413efcc2-770c-4ff0-9d11-a36d393b65e8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:20:12.068747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:12.068747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:12.068747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:12.068747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:12.068747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:12.068747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:12.068747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:12.068747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:12.068747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:12.068747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:12.068747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:12.068747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:12.068747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:12.068747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:12.068747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:12.069750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:12.069750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:12.069750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:12.069750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:12.069750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:12.069750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:12.069750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:12.069750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:12.069750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:12.069750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:12.069750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:12.069750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:12.069750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:12.069750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:12.069750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:12.069750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:12.069750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:12.069750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:12.071166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:12.071166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:12.071166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:12.071166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:12.071166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:12.071166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:12.071166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:12.071166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:12.071166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:20:12.071166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:12.071166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 308/638] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Aug 29 20:20:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:12.071782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413efcc2-770c-4ff0-9d11-a36d393b65e8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:12.095341 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a14c1-15e0-43e7-9a29-8eba6dc9a4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:12.096743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a14c1-15e0-43e7-9a29-8eba6dc9a4ca None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:12.097211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a14c1-15e0-43e7-9a29-8eba6dc9a4ca None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:12.098004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a14c1-15e0-43e7-9a29-8eba6dc9a4ca None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:12.099226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a14c1-15e0-43e7-9a29-8eba6dc9a4ca None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:12.099719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a14c1-15e0-43e7-9a29-8eba6dc9a4ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:12.101994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a14c1-15e0-43e7-9a29-8eba6dc9a4ca None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:12.128571 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 331/639] 10.222.0.18 () {56 vars in 1137 bytes} [Mon Aug 29 20:20:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:12.129801 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a14c1-15e0-43e7-9a29-8eba6dc9a4ca None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:12.179865 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:12.203256 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413efcc2-770c-4ff0-9d11-a36d393b65e8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:12.241826 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a6a40-97e7-43ac-8464-13fa40684b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:12.242453 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6a40-97e7-43ac-8464-13fa40684b73 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:12.242733 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6a40-97e7-43ac-8464-13fa40684b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:12.243005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6a40-97e7-43ac-8464-13fa40684b73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:12.253097 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a6a40-97e7-43ac-8464-13fa40684b73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:12.257237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a14c1-15e0-43e7-9a29-8eba6dc9a4ca None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:12.258357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-539a6a40-97e7-43ac-8464-13fa40684b73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:12.258357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:12.258357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:12.258357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:12.258357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:12.258357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:12.258357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:12.258357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:12.258357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:12.258357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:12.258357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:12.258357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:12.258357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:12.258357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:12.258357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:12.258357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:12.259247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:12.259247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:12.259247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:12.259247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:12.259247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:12.259247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:12.259247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:12.259247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:12.259247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:12.259247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:12.259247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:12.259247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:12.259247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:12.259247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:12.259247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:12.259247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:12.260096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:12.260096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:12.260096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:12.260096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:12.260096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:12.260096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:12.260096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:12.260096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:12.260096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:12.260096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:12.260096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:12.260096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:12.260096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 309/640] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:20:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:12.260777 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a6a40-97e7-43ac-8464-13fa40684b73 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:12.274577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:12.299003 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f0dd3-bc5b-45d7-a825-0a016f5859d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:12.299614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f0dd3-bc5b-45d7-a825-0a016f5859d1 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:12.299901 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f0dd3-bc5b-45d7-a825-0a016f5859d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:12.300207 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f0dd3-bc5b-45d7-a825-0a016f5859d1 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:12.301058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f0dd3-bc5b-45d7-a825-0a016f5859d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:12.301366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f0dd3-bc5b-45d7-a825-0a016f5859d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:12.302910 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f0dd3-bc5b-45d7-a825-0a016f5859d1 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:12.324129 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 332/641] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Aug 29 20:20:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:12.325193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f0dd3-bc5b-45d7-a825-0a016f5859d1 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:12.400690 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a6a40-97e7-43ac-8464-13fa40684b73 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:12.436310 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92206f79-fd3b-42ba-989b-b8602367a339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:12.436917 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92206f79-fd3b-42ba-989b-b8602367a339 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:12.437110 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f0dd3-bc5b-45d7-a825-0a016f5859d1 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:12.437538 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92206f79-fd3b-42ba-989b-b8602367a339 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:12.437538 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92206f79-fd3b-42ba-989b-b8602367a339 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:12.449589 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:12.473843 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381d766-8c8d-4c6e-9dd6-e1c32d2c527d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:12.474476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381d766-8c8d-4c6e-9dd6-e1c32d2c527d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:12.474759 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381d766-8c8d-4c6e-9dd6-e1c32d2c527d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:12.475036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381d766-8c8d-4c6e-9dd6-e1c32d2c527d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:12.486635 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381d766-8c8d-4c6e-9dd6-e1c32d2c527d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:12.491532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-7381d766-8c8d-4c6e-9dd6-e1c32d2c527d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:12.491532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:12.491532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:12.491532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:12.491532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:12.491532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:12.491532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:12.491532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:12.491532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:12.491532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:12.491532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:12.491532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:12.491532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:12.491532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:12.491532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:12.491532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:12.492556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:12.492556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:12.492556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:12.492556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:12.492556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:12.492556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:12.492556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:12.492556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:12.492556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:12.492556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:12.492556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:12.492556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:12.492556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:12.492556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:12.492556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:12.492556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:12.493435 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:12.493435 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:12.493435 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:12.493435 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:12.493435 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:12.493435 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:12.493435 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:12.493435 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:12.493435 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:12.493435 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:12.493435 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:12.493435 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:12.493435 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 333/642] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:20:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:12.494141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381d766-8c8d-4c6e-9dd6-e1c32d2c527d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:12.521147 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92206f79-fd3b-42ba-989b-b8602367a339 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:12.522671 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92206f79-fd3b-42ba-989b-b8602367a339 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:12.532656 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:12.617395 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381d766-8c8d-4c6e-9dd6-e1c32d2c527d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:12.648071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 310/643] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:20:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:20:12.649257 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92206f79-fd3b-42ba-989b-b8602367a339 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:12.654631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8357b45-f3c9-425b-9653-1c6f49c3088c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:12.655188 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8357b45-f3c9-425b-9653-1c6f49c3088c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:12.655471 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8357b45-f3c9-425b-9653-1c6f49c3088c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:12.655747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8357b45-f3c9-425b-9653-1c6f49c3088c None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:12.656448 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8357b45-f3c9-425b-9653-1c6f49c3088c None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:12.656751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8357b45-f3c9-425b-9653-1c6f49c3088c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:12.658110 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8357b45-f3c9-425b-9653-1c6f49c3088c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:12.674181 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 334/644] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:20:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:12.675027 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8357b45-f3c9-425b-9653-1c6f49c3088c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:12.700364 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:12.761067 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92206f79-fd3b-42ba-989b-b8602367a339 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:12.793842 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:12.801065 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c1bc5-57aa-4e35-8665-3171f29f9ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:12.801672 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c1bc5-57aa-4e35-8665-3171f29f9ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:12.801976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c1bc5-57aa-4e35-8665-3171f29f9ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:12.802254 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c1bc5-57aa-4e35-8665-3171f29f9ec3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:12.804288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8357b45-f3c9-425b-9653-1c6f49c3088c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:12.850270 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26231357-8e77-4529-a453-ac416e2528da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:12.851008 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26231357-8e77-4529-a453-ac416e2528da None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:12.851290 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26231357-8e77-4529-a453-ac416e2528da None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:12.851566 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26231357-8e77-4529-a453-ac416e2528da None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:12.852265 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26231357-8e77-4529-a453-ac416e2528da None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:12.852573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26231357-8e77-4529-a453-ac416e2528da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:12.853937 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26231357-8e77-4529-a453-ac416e2528da None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:12.863800 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 335/645] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:20:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:12.864791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26231357-8e77-4529-a453-ac416e2528da None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:12.894999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c1bc5-57aa-4e35-8665-3171f29f9ec3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:12.896780 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c1bc5-57aa-4e35-8665-3171f29f9ec3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:12.942910 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:12.980065 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26231357-8e77-4529-a453-ac416e2528da None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:13.003786 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 311/646] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:20:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:20:13.004664 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c1bc5-57aa-4e35-8665-3171f29f9ec3 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:13.014473 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a572b8-1000-49fc-9729-856f59e41e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:13.015111 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a572b8-1000-49fc-9729-856f59e41e24 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:13.015394 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a572b8-1000-49fc-9729-856f59e41e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:13.015669 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a572b8-1000-49fc-9729-856f59e41e24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:13.026151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a572b8-1000-49fc-9729-856f59e41e24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:13.027548 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a572b8-1000-49fc-9729-856f59e41e24 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:13.041791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-f2a572b8-1000-49fc-9729-856f59e41e24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:13.041791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:13.041791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:13.041791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:13.041791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:13.041791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:13.041791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:13.041791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:13.041791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:13.041791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:13.041791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:13.041791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:13.041791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:13.041791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:13.041791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:13.041791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:13.044076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:13.044076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:13.044076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:13.044076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:13.044076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:13.044076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:13.044076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:13.044076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:13.044076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:13.044076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:13.044076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:13.044076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:13.044076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:13.044076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:13.044076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:13.044076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:13.044076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:13.046122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:13.046122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:13.046122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:13.046122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 336/647] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:13.046122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a572b8-1000-49fc-9729-856f59e41e24 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:13.057484 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:13.105509 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c1bc5-57aa-4e35-8665-3171f29f9ec3 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:13.139409 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16da96f-e51e-4381-9740-2721a304dad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:13.140058 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16da96f-e51e-4381-9740-2721a304dad3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:13.140343 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16da96f-e51e-4381-9740-2721a304dad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:13.140611 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16da96f-e51e-4381-9740-2721a304dad3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:13.148553 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:13.149107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a572b8-1000-49fc-9729-856f59e41e24 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:13.150670 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16da96f-e51e-4381-9740-2721a304dad3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:13.152755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16da96f-e51e-4381-9740-2721a304dad3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:13.167725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-d16da96f-e51e-4381-9740-2721a304dad3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:13.167725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:13.167725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:13.167725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:13.167725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:13.167725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:13.167725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:13.167725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:13.167725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:13.167725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:13.167725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:13.167725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:13.167725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:13.167725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:13.167725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:13.167725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:13.169216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:13.169216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:13.169216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:13.169216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:13.169216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:13.169216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:13.169216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:13.169216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:13.169216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:13.169216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:13.169216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:13.169216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:13.169216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:13.169216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:13.169216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:13.169216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:13.169216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:13.170120 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:13.170120 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:13.170120 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:13.170120 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 312/648] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:13.170917 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bd748-83ac-44f0-a031-6e497596ce70 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:13.171276 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bd748-83ac-44f0-a031-6e497596ce70 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:13.171467 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bd748-83ac-44f0-a031-6e497596ce70 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:13.172963 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 313/649] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:13.173634 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949bd748-83ac-44f0-a031-6e497596ce70 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:13.189478 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258de3d3-5a04-4023-9542-d59f8e9855fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:13.190110 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258de3d3-5a04-4023-9542-d59f8e9855fb None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:13.190383 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258de3d3-5a04-4023-9542-d59f8e9855fb None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:13.190653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258de3d3-5a04-4023-9542-d59f8e9855fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:13.201003 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258de3d3-5a04-4023-9542-d59f8e9855fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:13.202360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258de3d3-5a04-4023-9542-d59f8e9855fb None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:13.217477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-258de3d3-5a04-4023-9542-d59f8e9855fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:13.217477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:13.217477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:13.217477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:13.217477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:13.217477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:13.217477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:13.217477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:13.217477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:13.217477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:13.217477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:13.217477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:13.217477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:13.217477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:13.217477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:13.217477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:13.218476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:13.218476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:13.218476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:13.218476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:13.218476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:13.218476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:13.218476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:13.218476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:13.218476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:13.218476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:13.218476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:13.218476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:13.218476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:13.218476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:13.218476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:13.218476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:13.218476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:13.219333 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:13.219333 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:13.219333 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:13.219333 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 337/650] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:13.219333 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258de3d3-5a04-4023-9542-d59f8e9855fb None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:13.278033 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:13.278545 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949bd748-83ac-44f0-a031-6e497596ce70 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:13.313847 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7757e09-2ac3-4ff1-b31e-a458ae03291e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:13.314472 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7757e09-2ac3-4ff1-b31e-a458ae03291e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:13.314745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7757e09-2ac3-4ff1-b31e-a458ae03291e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:13.315011 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7757e09-2ac3-4ff1-b31e-a458ae03291e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:13.323256 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258de3d3-5a04-4023-9542-d59f8e9855fb None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:13.326253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7757e09-2ac3-4ff1-b31e-a458ae03291e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:13.327701 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7757e09-2ac3-4ff1-b31e-a458ae03291e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:13.342225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-c7757e09-2ac3-4ff1-b31e-a458ae03291e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:13.342225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:13.342225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:13.342225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:13.342225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:13.342225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:13.342225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:13.342225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:13.342225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:13.342225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:13.342225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:13.342225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:13.342225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:13.342225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:13.342225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:13.342225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:13.343184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:13.343184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:13.343184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:13.343184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:13.343184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:13.343184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:13.343184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:13.343184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:13.343184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:13.343184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:13.343184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:13.343184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:13.343184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:13.343184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:13.343184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:13.343184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:13.343184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:13.344087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:13.344087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:13.344087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:13.344087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 314/651] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:13.344087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7757e09-2ac3-4ff1-b31e-a458ae03291e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:13.363328 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b300b8-cb4d-4157-a822-bbf4b722336b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:13.363960 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b300b8-cb4d-4157-a822-bbf4b722336b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:13.364243 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b300b8-cb4d-4157-a822-bbf4b722336b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:13.364517 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b300b8-cb4d-4157-a822-bbf4b722336b None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:13.365156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b300b8-cb4d-4157-a822-bbf4b722336b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:13.365459 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b300b8-cb4d-4157-a822-bbf4b722336b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:13.366807 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b300b8-cb4d-4157-a822-bbf4b722336b None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:13.378959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 338/652] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:13.380144 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6991a3a-b8de-4801-9c3f-08beaeeed5c2 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:13.380425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6991a3a-b8de-4801-9c3f-08beaeeed5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:13.380698 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6991a3a-b8de-4801-9c3f-08beaeeed5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:13.390865 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:13.398527 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f6991a3a-b8de-4801-9c3f-08beaeeed5c2 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:13.399711 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f6991a3a-b8de-4801-9c3f-08beaeeed5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:13.455845 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7757e09-2ac3-4ff1-b31e-a458ae03291e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:13.466700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 339/653] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:20:13.467617 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6991a3a-b8de-4801-9c3f-08beaeeed5c2 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:13.490391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d625df-fe54-4961-a951-af376635afaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:13.490988 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d625df-fe54-4961-a951-af376635afaf None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:13.491289 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d625df-fe54-4961-a951-af376635afaf None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:13.491557 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d625df-fe54-4961-a951-af376635afaf None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:13.492224 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d625df-fe54-4961-a951-af376635afaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:13.492521 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d625df-fe54-4961-a951-af376635afaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:13.493872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d625df-fe54-4961-a951-af376635afaf None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:13.506004 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 315/654] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:13.506821 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d625df-fe54-4961-a951-af376635afaf None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:13.521608 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:13.582099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6991a3a-b8de-4801-9c3f-08beaeeed5c2 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:13.620441 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2829a5ab-ca16-4d4e-880e-2b1b0fe07998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:13.621062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829a5ab-ca16-4d4e-880e-2b1b0fe07998 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:13.622207 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829a5ab-ca16-4d4e-880e-2b1b0fe07998 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:13.622494 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829a5ab-ca16-4d4e-880e-2b1b0fe07998 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:13.623264 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2829a5ab-ca16-4d4e-880e-2b1b0fe07998 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:13.623593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2829a5ab-ca16-4d4e-880e-2b1b0fe07998 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:13.625100 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2829a5ab-ca16-4d4e-880e-2b1b0fe07998 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:13.628737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d625df-fe54-4961-a951-af376635afaf None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:13.631076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:13.645273 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 340/655] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:13.646179 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2829a5ab-ca16-4d4e-880e-2b1b0fe07998 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:13.675274 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a7e95-4799-463f-99cf-777a3b1549ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:13.675888 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a7e95-4799-463f-99cf-777a3b1549ec None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:13.676209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a7e95-4799-463f-99cf-777a3b1549ec None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:13.676473 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a7e95-4799-463f-99cf-777a3b1549ec None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:13.677122 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a7e95-4799-463f-99cf-777a3b1549ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:13.677422 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a7e95-4799-463f-99cf-777a3b1549ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:13.678795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a7e95-4799-463f-99cf-777a3b1549ec None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:13.690602 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 316/656] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:13.691324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a7e95-4799-463f-99cf-777a3b1549ec None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:13.770203 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2829a5ab-ca16-4d4e-880e-2b1b0fe07998 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:13.779690 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:13.801475 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a7e95-4799-463f-99cf-777a3b1549ec None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:13.817656 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9060f-558d-4051-a023-2762d9b0833d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:13.818658 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9060f-558d-4051-a023-2762d9b0833d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:13.819123 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9060f-558d-4051-a023-2762d9b0833d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:13.819574 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9060f-558d-4051-a023-2762d9b0833d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:13.832047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe9060f-558d-4051-a023-2762d9b0833d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:13.834559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe9060f-558d-4051-a023-2762d9b0833d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:13.835304 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07df00-c8c6-4001-8675-3af5166c5569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:13.835978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07df00-c8c6-4001-8675-3af5166c5569 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:13.836260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07df00-c8c6-4001-8675-3af5166c5569 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:13.836527 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07df00-c8c6-4001-8675-3af5166c5569 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:13.844611 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07df00-c8c6-4001-8675-3af5166c5569 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:13.846021 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07df00-c8c6-4001-8675-3af5166c5569 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:13.854950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-3fe9060f-558d-4051-a023-2762d9b0833d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:13.854950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:13.854950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:13.854950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:13.854950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:13.854950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:13.854950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:13.854950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:13.854950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:13.854950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:13.854950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:13.854950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:13.854950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:13.854950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:13.854950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:13.856114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:13.856114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:13.856114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:13.856114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:13.856114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:13.856114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:13.856114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:13.856114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:13.856114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:13.856114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:13.856114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:13.856114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:13.856114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:13.856114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:13.856114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:13.856114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:13.856114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:13.857320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:13.857320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:13.857320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:13.857320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:13.857320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:13.857320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:13.857320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:13.857320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:13.857320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 341/657] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:13.858276 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9060f-558d-4051-a023-2762d9b0833d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:13.861164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-7a07df00-c8c6-4001-8675-3af5166c5569 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 29 20:20:13.861164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:13.861164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:13.861164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:13.861164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:13.861164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:13.861164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:13.861164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:13.861164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:13.861164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:13.861164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:13.861164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:13.861164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:13.861164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:13.861164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:13.861164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:13.862085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:13.862085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:13.862085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:13.862085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:13.862085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:13.862085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:13.862085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:13.862085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:13.862085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:13.862085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:13.862085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:13.862085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:13.862085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:13.862085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:13.862085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:13.862085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:13.862085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:13.862085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:13.867976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:13.867976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:13.867976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:13.867976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:13.867976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:13.867976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:13.867976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:13.867976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:13.867976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 29 20:20:13.867976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:13.867976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 317/658] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:20:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:13.867976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07df00-c8c6-4001-8675-3af5166c5569 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:13.891356 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:13.967475 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07df00-c8c6-4001-8675-3af5166c5569 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:13.971154 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9060f-558d-4051-a023-2762d9b0833d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:14.003476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd2e52-0205-46c5-9f4a-8db595b90ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:14.004130 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd2e52-0205-46c5-9f4a-8db595b90ced None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:14.004403 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd2e52-0205-46c5-9f4a-8db595b90ced None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:14.004670 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fd2e52-0205-46c5-9f4a-8db595b90ced None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:14.008052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837740fc-4a2a-49be-b08f-4c23639b3e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:14.008687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837740fc-4a2a-49be-b08f-4c23639b3e91 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:14.008967 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837740fc-4a2a-49be-b08f-4c23639b3e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:14.009238 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837740fc-4a2a-49be-b08f-4c23639b3e91 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:14.011323 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd2e52-0205-46c5-9f4a-8db595b90ced None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:14.013370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fd2e52-0205-46c5-9f4a-8db595b90ced None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:14.018340 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837740fc-4a2a-49be-b08f-4c23639b3e91 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:14.019768 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837740fc-4a2a-49be-b08f-4c23639b3e91 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:14.028007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-83fd2e52-0205-46c5-9f4a-8db595b90ced None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:14.028007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:14.028007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:14.028007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:14.028007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:14.028007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:14.028007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:14.028007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:14.028007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:14.028007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:14.028007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:14.028007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:14.028007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:14.028007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:14.028007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:14.028909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:14.028909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:14.028909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:14.028909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:14.028909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:14.028909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:14.028909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:14.028909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:14.028909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:14.028909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:14.028909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:14.028909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:14.028909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:14.028909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:14.028909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:14.028909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:14.028909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:14.029779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:14.029779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:14.029779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:14.029779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:14.029779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:14.029779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:14.029779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:14.029779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:14.029779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 318/659] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:14.030649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd2e52-0205-46c5-9f4a-8db595b90ced None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:14.031421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-837740fc-4a2a-49be-b08f-4c23639b3e91 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 20:20:14.031421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:14.031421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:14.031421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:14.031421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:14.031421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:14.031421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:14.031421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:14.031421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:14.031421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:14.031421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:14.031421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:14.031421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:14.031421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:14.031421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:14.031421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:14.032750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:14.032750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:14.032750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:14.032750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:14.032750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:14.032750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:14.032750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:14.032750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:14.032750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:14.032750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:14.032750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:14.032750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:14.032750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:14.032750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:14.032750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:14.032750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:14.032750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:14.032750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:14.033662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:14.033662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:14.033662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:14.033662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:14.033662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:14.033662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:14.033662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:14.033662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:14.033662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 29 20:20:14.033662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:14.033662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 342/660] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:20:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:14.034262 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837740fc-4a2a-49be-b08f-4c23639b3e91 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:14.070411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:14.148736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:14.155119 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837740fc-4a2a-49be-b08f-4c23639b3e91 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:14.165038 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fd2e52-0205-46c5-9f4a-8db595b90ced None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:14.192057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7d01f-b43c-483d-8a43-625dfefbef0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:14.192674 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7d01f-b43c-483d-8a43-625dfefbef0b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:14.192952 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7d01f-b43c-483d-8a43-625dfefbef0b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:14.193225 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7d01f-b43c-483d-8a43-625dfefbef0b None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:14.193959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7d01f-b43c-483d-8a43-625dfefbef0b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:14.194265 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7d01f-b43c-483d-8a43-625dfefbef0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:14.195690 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7d01f-b43c-483d-8a43-625dfefbef0b None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:14.200356 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c6560-f74d-42bc-8243-add41817498f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:14.201040 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c6560-f74d-42bc-8243-add41817498f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:14.201325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c6560-f74d-42bc-8243-add41817498f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:14.201600 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c6560-f74d-42bc-8243-add41817498f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:14.212837 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307c6560-f74d-42bc-8243-add41817498f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:14.219972 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 343/661] 10.222.0.18 () {56 vars in 1137 bytes} [Mon Aug 29 20:20:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:14.219972 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7d01f-b43c-483d-8a43-625dfefbef0b None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:14.220198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-307c6560-f74d-42bc-8243-add41817498f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:14.220198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:14.220198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:14.220198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:14.220198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:14.220198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:14.220198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:14.220198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:14.220198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:14.220198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:14.220198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:14.220198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:14.220198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:14.220198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:14.220198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:14.220198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:14.221299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:14.221299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:14.221299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:14.221299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:14.221299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:14.221299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:14.221299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:14.221299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:14.221299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:14.221299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:14.221299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:14.221299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:14.221299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:14.221299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:14.221299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:14.221299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:14.222324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:14.222324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:14.222324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:14.222324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:14.222324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:14.222324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:14.222324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:14.222324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:14.222324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:14.222324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:14.222324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:14.222324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:14.222324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 319/662] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:20:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:14.222324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c6560-f74d-42bc-8243-add41817498f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:14.330650 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:14.346311 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7d01f-b43c-483d-8a43-625dfefbef0b None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:14.354205 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c6560-f74d-42bc-8243-add41817498f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:14.380545 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a5abc3-5f2b-432f-ae66-19a1af24de4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:14.381156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5abc3-5f2b-432f-ae66-19a1af24de4d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:14.381431 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5abc3-5f2b-432f-ae66-19a1af24de4d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:14.381708 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5abc3-5f2b-432f-ae66-19a1af24de4d None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:14.382394 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a5abc3-5f2b-432f-ae66-19a1af24de4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:14.382698 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a5abc3-5f2b-432f-ae66-19a1af24de4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:14.384135 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a5abc3-5f2b-432f-ae66-19a1af24de4d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:14.388263 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aad442-b8fb-4e1d-9770-c56babaf0038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:14.388871 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aad442-b8fb-4e1d-9770-c56babaf0038 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:14.389150 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aad442-b8fb-4e1d-9770-c56babaf0038 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:14.389420 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aad442-b8fb-4e1d-9770-c56babaf0038 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:14.390108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aad442-b8fb-4e1d-9770-c56babaf0038 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:14.390412 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aad442-b8fb-4e1d-9770-c56babaf0038 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:14.391819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aad442-b8fb-4e1d-9770-c56babaf0038 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:14.395410 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 344/663] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:14.396156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a5abc3-5f2b-432f-ae66-19a1af24de4d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:14.405312 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 320/664] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:14.406132 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aad442-b8fb-4e1d-9770-c56babaf0038 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:14.410359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:14.498343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a5abc3-5f2b-432f-ae66-19a1af24de4d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:14.506954 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aad442-b8fb-4e1d-9770-c56babaf0038 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:14.531878 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cdcfb-10f3-4269-8559-010c049d457c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:14.532527 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cdcfb-10f3-4269-8559-010c049d457c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:14.532831 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cdcfb-10f3-4269-8559-010c049d457c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:14.533108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cdcfb-10f3-4269-8559-010c049d457c None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:14.533855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98cdcfb-10f3-4269-8559-010c049d457c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:14.534157 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98cdcfb-10f3-4269-8559-010c049d457c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:14.535562 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98cdcfb-10f3-4269-8559-010c049d457c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:14.540908 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ed4b5-c027-4ee8-bcb7-7cd8336c8944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:14.541510 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ed4b5-c027-4ee8-bcb7-7cd8336c8944 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:14.541789 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ed4b5-c027-4ee8-bcb7-7cd8336c8944 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:14.542059 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ed4b5-c027-4ee8-bcb7-7cd8336c8944 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:14.553470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 345/665] 10.222.0.18 () {56 vars in 1139 bytes} [Mon Aug 29 20:20:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:14.554492 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cdcfb-10f3-4269-8559-010c049d457c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:14.557827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:14.616005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ed4b5-c027-4ee8-bcb7-7cd8336c8944 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:14.618594 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ed4b5-c027-4ee8-bcb7-7cd8336c8944 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:14.622416 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:14.659445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cdcfb-10f3-4269-8559-010c049d457c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:14.693382 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a851105-4094-402c-ac76-9e2d4f5a228f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:14.693990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a851105-4094-402c-ac76-9e2d4f5a228f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:14.694275 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a851105-4094-402c-ac76-9e2d4f5a228f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:14.694545 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a851105-4094-402c-ac76-9e2d4f5a228f None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:14.696154 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a851105-4094-402c-ac76-9e2d4f5a228f None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:14.696154 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a851105-4094-402c-ac76-9e2d4f5a228f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:14.696885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a851105-4094-402c-ac76-9e2d4f5a228f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:14.705377 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 346/666] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:20:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:14.706175 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a851105-4094-402c-ac76-9e2d4f5a228f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:14.724084 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 321/667] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:20:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:20:14.724932 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ed4b5-c027-4ee8-bcb7-7cd8336c8944 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:14.777286 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:14.813463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a851105-4094-402c-ac76-9e2d4f5a228f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:14.825961 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ed4b5-c027-4ee8-bcb7-7cd8336c8944 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:14.828365 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:14.848633 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff2170-328e-46cc-88bc-284cb7f3d920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:14.849247 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff2170-328e-46cc-88bc-284cb7f3d920 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:14.849530 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff2170-328e-46cc-88bc-284cb7f3d920 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:14.849813 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff2170-328e-46cc-88bc-284cb7f3d920 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:14.850553 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff2170-328e-46cc-88bc-284cb7f3d920 None admin] RBAC: Adding query filter params (role.id=04bc48cc5b6842029b93a845d6744985, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=85624cea63af40fda09f57ddf2cd5c3a) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:14.850874 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff2170-328e-46cc-88bc-284cb7f3d920 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:14.852324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff2170-328e-46cc-88bc-284cb7f3d920 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:14.858290 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 347/668] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:20:14 2022] GET /identity/v3/role_assignments?user.id=85624cea63af40fda09f57ddf2cd5c3a&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=04bc48cc5b6842029b93a845d6744985 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:14.859103 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff2170-328e-46cc-88bc-284cb7f3d920 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:14.869602 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701931ea-97dc-4fa5-b928-0f8e9785988b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:14.871207 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701931ea-97dc-4fa5-b928-0f8e9785988b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:14.872109 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701931ea-97dc-4fa5-b928-0f8e9785988b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:14.872336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701931ea-97dc-4fa5-b928-0f8e9785988b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:14.873118 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701931ea-97dc-4fa5-b928-0f8e9785988b None admin] RBAC: Adding query filter params (role.id=04bc48cc5b6842029b93a845d6744985, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=c0c44754a1c04a2b87081aba6cb509d3) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:14.873430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701931ea-97dc-4fa5-b928-0f8e9785988b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:14.874946 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701931ea-97dc-4fa5-b928-0f8e9785988b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:14.882521 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 322/669] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:20:14 2022] GET /identity/v3/role_assignments?user.id=c0c44754a1c04a2b87081aba6cb509d3&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=04bc48cc5b6842029b93a845d6744985 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:14.883221 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701931ea-97dc-4fa5-b928-0f8e9785988b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:15.030836 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff2170-328e-46cc-88bc-284cb7f3d920 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:15.041925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701931ea-97dc-4fa5-b928-0f8e9785988b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:15.066258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:15.072583 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6fbed-8a22-4e24-ac34-c044e7c98fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:15.073872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6fbed-8a22-4e24-ac34-c044e7c98fdf None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:15.073872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6fbed-8a22-4e24-ac34-c044e7c98fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:15.074094 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6fbed-8a22-4e24-ac34-c044e7c98fdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:15.085726 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b4e92-3c73-4986-8400-db2fadad3b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:15.086745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b4e92-3c73-4986-8400-db2fadad3b96 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:15.087186 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b4e92-3c73-4986-8400-db2fadad3b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:15.087512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b4e92-3c73-4986-8400-db2fadad3b96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:15.100184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b4e92-3c73-4986-8400-db2fadad3b96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:15.101694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b4e92-3c73-4986-8400-db2fadad3b96 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:15.119294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-eb0b4e92-3c73-4986-8400-db2fadad3b96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:15.119294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:15.119294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:15.119294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:15.119294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:15.119294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:15.119294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:15.119294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:15.119294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:15.119294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:15.119294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:15.119294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:15.119294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:15.119294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:15.119294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:15.119294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:15.120493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:15.120493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:15.120493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:15.120493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:15.120493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:15.120493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:15.120493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:15.120493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:15.120493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:15.120493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:15.120493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:15.120493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:15.120493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:15.120493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:15.120493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:15.120493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:15.120493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:15.121658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:15.121658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:15.121658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:15.121658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 323/670] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:15.121658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b4e92-3c73-4986-8400-db2fadad3b96 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:15.157290 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:15.162585 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6fbed-8a22-4e24-ac34-c044e7c98fdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:15.164438 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6fbed-8a22-4e24-ac34-c044e7c98fdf None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:15.258098 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b4e92-3c73-4986-8400-db2fadad3b96 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:15.275500 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 348/671] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:20:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:20:15.280581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6fbed-8a22-4e24-ac34-c044e7c98fdf None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:15.309971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b5a1d-e99e-4589-93c8-4c456f7cc636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:15.310832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b5a1d-e99e-4589-93c8-4c456f7cc636 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:15.311253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b5a1d-e99e-4589-93c8-4c456f7cc636 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:15.311676 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b5a1d-e99e-4589-93c8-4c456f7cc636 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:15.321107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:15.329899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b5a1d-e99e-4589-93c8-4c456f7cc636 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:15.331623 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b5a1d-e99e-4589-93c8-4c456f7cc636 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:15.352133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-543b5a1d-e99e-4589-93c8-4c456f7cc636 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:15.352133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:15.352133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:15.352133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:15.352133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:15.352133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:15.352133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:15.352133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:15.352133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:15.352133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:15.352133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:15.352133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:15.352133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:15.352133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:15.352133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:15.352133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:15.353297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:15.353297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:15.353297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:15.353297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:15.353297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:15.353297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:15.353297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:15.353297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:15.353297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:15.353297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:15.353297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:15.353297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:15.353297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:15.353297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:15.353297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:15.353297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:15.353297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:15.354433 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:15.354433 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:15.354433 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:15.356915 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 324/672] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:15.361659 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b5a1d-e99e-4589-93c8-4c456f7cc636 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:15.420685 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6fbed-8a22-4e24-ac34-c044e7c98fdf None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:15.437878 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:15.470801 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a808ea3-ccc4-478b-b8e0-0f4b63320a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:15.472058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a808ea3-ccc4-478b-b8e0-0f4b63320a1e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:15.472951 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a808ea3-ccc4-478b-b8e0-0f4b63320a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:15.473441 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a808ea3-ccc4-478b-b8e0-0f4b63320a1e None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:15.474481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a808ea3-ccc4-478b-b8e0-0f4b63320a1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:15.474936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a808ea3-ccc4-478b-b8e0-0f4b63320a1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:15.476100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b5a1d-e99e-4589-93c8-4c456f7cc636 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:15.476654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a808ea3-ccc4-478b-b8e0-0f4b63320a1e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:15.497677 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 349/673] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:15.499290 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a808ea3-ccc4-478b-b8e0-0f4b63320a1e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:15.523252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7ca75-ceaa-47d4-b3f8-44ce3e43e623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:15.524272 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7ca75-ceaa-47d4-b3f8-44ce3e43e623 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:15.524794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7ca75-ceaa-47d4-b3f8-44ce3e43e623 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:15.525828 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7ca75-ceaa-47d4-b3f8-44ce3e43e623 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:15.541905 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7ca75-ceaa-47d4-b3f8-44ce3e43e623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:15.543713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de7ca75-ceaa-47d4-b3f8-44ce3e43e623 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:15.563967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-9de7ca75-ceaa-47d4-b3f8-44ce3e43e623 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:15.563967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:15.563967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:15.563967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:15.563967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:15.563967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:15.563967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:15.563967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:15.563967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:15.563967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:15.563967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:15.563967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:15.563967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:15.563967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:15.563967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:15.563967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:15.564972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:15.564972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:15.564972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:15.564972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:15.564972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:15.564972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:15.564972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:15.564972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:15.564972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:15.564972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:15.564972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:15.564972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:15.564972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:15.564972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:15.564972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:15.564972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:15.564972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:15.565883 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:15.565883 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:15.565883 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:15.568702 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 325/674] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:15.569792 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7ca75-ceaa-47d4-b3f8-44ce3e43e623 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:15.629560 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:15.643926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a808ea3-ccc4-478b-b8e0-0f4b63320a1e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:15.680829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0981a8-6315-447e-9d00-7cd0f326a8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:15.681541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0981a8-6315-447e-9d00-7cd0f326a8e4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:15.681828 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0981a8-6315-447e-9d00-7cd0f326a8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:15.682101 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0981a8-6315-447e-9d00-7cd0f326a8e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:15.682750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0981a8-6315-447e-9d00-7cd0f326a8e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:15.683053 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0981a8-6315-447e-9d00-7cd0f326a8e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:15.685417 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0981a8-6315-447e-9d00-7cd0f326a8e4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:15.692083 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7ca75-ceaa-47d4-b3f8-44ce3e43e623 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:15.704787 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 350/675] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:15.704787 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0981a8-6315-447e-9d00-7cd0f326a8e4 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:15.734384 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:15.746791 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd67da-e9ec-427d-9abb-28702ca802e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:15.747702 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd67da-e9ec-427d-9abb-28702ca802e2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:15.748231 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd67da-e9ec-427d-9abb-28702ca802e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:15.748733 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd67da-e9ec-427d-9abb-28702ca802e2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:15.757207 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd67da-e9ec-427d-9abb-28702ca802e2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:15.759018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd67da-e9ec-427d-9abb-28702ca802e2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:15.771803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-22fd67da-e9ec-427d-9abb-28702ca802e2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:20:15.771803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:15.771803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:15.771803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:15.771803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:15.771803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:15.771803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:15.771803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:15.771803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:15.771803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:15.771803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:15.771803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:15.771803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:15.771803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:15.771803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:15.771803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:15.772921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:15.772921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:15.772921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:15.772921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:15.772921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:15.772921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:15.772921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:15.772921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:15.772921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:15.772921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:15.772921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:15.772921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:15.772921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:15.772921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:15.772921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:15.772921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:15.772921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:15.772921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:15.773846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:15.773846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:15.773846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:15.773846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:15.773846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:15.773846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:15.773846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:15.773846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:15.773846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:20:15.773846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:15.780338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 326/676] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:20:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:15.781401 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd67da-e9ec-427d-9abb-28702ca802e2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:15.816513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0981a8-6315-447e-9d00-7cd0f326a8e4 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:15.853509 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdd674-ebba-45e0-9825-0d71b6f34f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:15.853952 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdd674-ebba-45e0-9825-0d71b6f34f23 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:15.854235 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdd674-ebba-45e0-9825-0d71b6f34f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:15.854505 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdd674-ebba-45e0-9825-0d71b6f34f23 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:15.855144 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bdd674-ebba-45e0-9825-0d71b6f34f23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:15.855442 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bdd674-ebba-45e0-9825-0d71b6f34f23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:15.857396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bdd674-ebba-45e0-9825-0d71b6f34f23 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:15.874675 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 351/677] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:15.875702 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdd674-ebba-45e0-9825-0d71b6f34f23 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:15.889241 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:15.902609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd67da-e9ec-427d-9abb-28702ca802e2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:15.942781 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a307cf-76cc-4a2e-9a0f-2162d7a70086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:15.943413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a307cf-76cc-4a2e-9a0f-2162d7a70086 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:15.943682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a307cf-76cc-4a2e-9a0f-2162d7a70086 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:15.943974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a307cf-76cc-4a2e-9a0f-2162d7a70086 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:15.950678 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a307cf-76cc-4a2e-9a0f-2162d7a70086 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:15.952212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a307cf-76cc-4a2e-9a0f-2162d7a70086 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:15.963868 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-f5a307cf-76cc-4a2e-9a0f-2162d7a70086 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:15.963868 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:15.963868 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:15.963868 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:15.963868 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:15.963868 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:15.963868 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:15.963868 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:15.963868 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:15.963868 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:15.963868 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:15.963868 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:15.963868 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:15.963868 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:15.963868 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:15.964973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:15.964973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:15.964973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:15.964973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:15.964973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:15.964973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:15.964973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:15.964973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:15.964973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:15.964973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:15.964973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:15.964973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:15.964973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:15.964973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:15.964973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:15.964973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:15.964973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:15.966418 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:15.966418 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:15.966418 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:15.966418 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:15.966418 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:15.966418 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:15.966418 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:15.966418 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:15.969547 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 327/678] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:15.970445 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a307cf-76cc-4a2e-9a0f-2162d7a70086 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:16.012793 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdd674-ebba-45e0-9825-0d71b6f34f23 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:16.052314 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:16.057178 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81282b1-287a-4b2d-9d31-b4821e17e100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:16.058186 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81282b1-287a-4b2d-9d31-b4821e17e100 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:16.058680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81282b1-287a-4b2d-9d31-b4821e17e100 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:16.059256 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81282b1-287a-4b2d-9d31-b4821e17e100 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:16.060767 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81282b1-287a-4b2d-9d31-b4821e17e100 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:16.061344 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81282b1-287a-4b2d-9d31-b4821e17e100 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:16.063041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81282b1-287a-4b2d-9d31-b4821e17e100 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:16.084438 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 352/679] 10.222.0.18 () {56 vars in 1135 bytes} [Mon Aug 29 20:20:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:16.085724 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81282b1-287a-4b2d-9d31-b4821e17e100 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:16.096655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a307cf-76cc-4a2e-9a0f-2162d7a70086 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:16.136564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af77b1-f1c0-4d27-81ba-30be4583dcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:16.137719 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af77b1-f1c0-4d27-81ba-30be4583dcdd None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:16.138000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af77b1-f1c0-4d27-81ba-30be4583dcdd None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:16.138267 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af77b1-f1c0-4d27-81ba-30be4583dcdd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:16.145757 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af77b1-f1c0-4d27-81ba-30be4583dcdd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:16.147449 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af77b1-f1c0-4d27-81ba-30be4583dcdd None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:16.160372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-35af77b1-f1c0-4d27-81ba-30be4583dcdd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:16.160372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:16.160372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:16.160372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:16.160372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:16.160372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:16.160372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:16.160372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:16.160372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:16.160372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:16.160372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:16.160372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:16.160372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:16.160372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:16.160372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:16.161345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:16.161345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:16.161345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:16.161345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:16.161345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:16.161345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:16.161345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:16.161345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:16.161345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:16.161345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:16.161345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:16.161345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:16.161345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:16.161345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:16.161345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:16.161345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:16.161345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:16.162991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:16.162991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:16.162991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:16.162991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:16.162991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:16.162991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:16.162991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:16.162991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:16.162991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 328/680] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:16.162991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af77b1-f1c0-4d27-81ba-30be4583dcdd None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:16.172921 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:16.208339 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81282b1-287a-4b2d-9d31-b4821e17e100 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:16.244770 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54811351-f737-4f7c-bb2e-785645d5e8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:16.245873 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54811351-f737-4f7c-bb2e-785645d5e8b1 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:16.246891 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54811351-f737-4f7c-bb2e-785645d5e8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:16.247381 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54811351-f737-4f7c-bb2e-785645d5e8b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:16.248401 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54811351-f737-4f7c-bb2e-785645d5e8b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:16.248928 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54811351-f737-4f7c-bb2e-785645d5e8b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:16.250733 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54811351-f737-4f7c-bb2e-785645d5e8b1 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:16.264858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 353/681] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:16.266036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54811351-f737-4f7c-bb2e-785645d5e8b1 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:16.277521 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af77b1-f1c0-4d27-81ba-30be4583dcdd None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:16.280305 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:16.323179 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d29b1-1a8f-443c-a161-c534bdfc150f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:16.323794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d29b1-1a8f-443c-a161-c534bdfc150f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:16.324104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d29b1-1a8f-443c-a161-c534bdfc150f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:16.324419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d29b1-1a8f-443c-a161-c534bdfc150f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:16.387824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54811351-f737-4f7c-bb2e-785645d5e8b1 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:16.398729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d29b1-1a8f-443c-a161-c534bdfc150f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:16.400213 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8d29b1-1a8f-443c-a161-c534bdfc150f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:16.418921 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:16.427946 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2c3d2-2178-4116-87a0-799a076f9dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:16.428938 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2c3d2-2178-4116-87a0-799a076f9dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:16.429430 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2c3d2-2178-4116-87a0-799a076f9dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:16.429903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2c3d2-2178-4116-87a0-799a076f9dd4 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:16.430920 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2c3d2-2178-4116-87a0-799a076f9dd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:16.431479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2c3d2-2178-4116-87a0-799a076f9dd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:16.433190 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2c3d2-2178-4116-87a0-799a076f9dd4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:16.447910 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 354/682] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:16.449172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2c3d2-2178-4116-87a0-799a076f9dd4 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:16.515309 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 329/683] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:20:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:20:16.516541 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d29b1-1a8f-443c-a161-c534bdfc150f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:16.519819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:16.568344 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2c3d2-2178-4116-87a0-799a076f9dd4 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:16.607393 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2034198-d927-4ff6-9e64-a82534512fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:16.608409 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2034198-d927-4ff6-9e64-a82534512fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:16.608903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2034198-d927-4ff6-9e64-a82534512fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:16.609518 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2034198-d927-4ff6-9e64-a82534512fb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:16.610606 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2034198-d927-4ff6-9e64-a82534512fb6 None admin] RBAC: Adding query filter params (role.id=04bc48cc5b6842029b93a845d6744985, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=5d022ed8d2ac43ae816aa396a8e2f17a) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:16.611128 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2034198-d927-4ff6-9e64-a82534512fb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:16.613056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2034198-d927-4ff6-9e64-a82534512fb6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:16.620389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 355/684] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:20:16 2022] GET /identity/v3/role_assignments?user.id=5d022ed8d2ac43ae816aa396a8e2f17a&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=04bc48cc5b6842029b93a845d6744985 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:16.622208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2034198-d927-4ff6-9e64-a82534512fb6 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:16.633822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d29b1-1a8f-443c-a161-c534bdfc150f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:16.704919 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb30688-339a-435c-a278-b7b9e963380d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:16.705451 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb30688-339a-435c-a278-b7b9e963380d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:16.705731 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb30688-339a-435c-a278-b7b9e963380d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:16.706000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb30688-339a-435c-a278-b7b9e963380d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:16.706739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb30688-339a-435c-a278-b7b9e963380d None admin] RBAC: Adding query filter params (role.id=04bc48cc5b6842029b93a845d6744985, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=d51e331a4c9c4f038bdb80f8c5b74e42) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:16.711957 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb30688-339a-435c-a278-b7b9e963380d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:16.711957 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb30688-339a-435c-a278-b7b9e963380d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:16.719978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 330/685] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:20:16 2022] GET /identity/v3/role_assignments?user.id=d51e331a4c9c4f038bdb80f8c5b74e42&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=04bc48cc5b6842029b93a845d6744985 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:16.733887 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:16.795508 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2034198-d927-4ff6-9e64-a82534512fb6 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:16.849394 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5ab89-53ef-4ff3-ac99-ed79fa4fb2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:16.850058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5ab89-53ef-4ff3-ac99-ed79fa4fb2ef None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:16.850394 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5ab89-53ef-4ff3-ac99-ed79fa4fb2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:16.850668 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5ab89-53ef-4ff3-ac99-ed79fa4fb2ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:16.870272 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5ab89-53ef-4ff3-ac99-ed79fa4fb2ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:16.872318 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5ab89-53ef-4ff3-ac99-ed79fa4fb2ef None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:16.889577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-a7e5ab89-53ef-4ff3-ac99-ed79fa4fb2ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:16.889577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:16.889577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:16.889577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:16.889577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:16.889577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:16.889577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:16.889577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:16.889577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:16.889577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:16.889577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:16.889577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:16.889577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:16.889577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:16.889577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:16.889577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:16.890584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:16.890584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:16.890584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:16.890584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:16.890584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:16.890584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:16.890584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:16.890584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:16.890584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:16.890584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:16.890584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:16.890584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:16.890584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:16.890584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:16.890584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:16.890584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:16.890584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:16.892008 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:16.892008 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:16.892008 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:16.892008 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 356/686] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:16.897074 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb30688-339a-435c-a278-b7b9e963380d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:16.976361 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:16.998255 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f2573-6863-4ec9-ae53-d9cf9664a996 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:16.998582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f2573-6863-4ec9-ae53-d9cf9664a996 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:16.998806 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f2573-6863-4ec9-ae53-d9cf9664a996 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:17.000581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 357/687] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:17.006521 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7589e1-53fc-4101-9b7d-9f99fc594917 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:17.006921 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7589e1-53fc-4101-9b7d-9f99fc594917 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:17.007336 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7589e1-53fc-4101-9b7d-9f99fc594917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:17.023534 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb30688-339a-435c-a278-b7b9e963380d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:17.032013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-9c7589e1-53fc-4101-9b7d-9f99fc594917 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:17.033392 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9c7589e1-53fc-4101-9b7d-9f99fc594917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:17.067897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5039c-a200-478e-8431-1d99dda3e50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:17.068715 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5039c-a200-478e-8431-1d99dda3e50f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:17.069106 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5039c-a200-478e-8431-1d99dda3e50f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:17.069487 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5039c-a200-478e-8431-1d99dda3e50f None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:17.070289 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5039c-a200-478e-8431-1d99dda3e50f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:17.071533 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5039c-a200-478e-8431-1d99dda3e50f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:17.072278 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5039c-a200-478e-8431-1d99dda3e50f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:17.090639 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 331/688] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:17.090639 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcde8d-9858-4d54-b677-6be845af9ebf None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:17.090639 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcde8d-9858-4d54-b677-6be845af9ebf None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:17.090639 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcde8d-9858-4d54-b677-6be845af9ebf None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:17.092265 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 332/689] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:17.097649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3383502-119f-4417-81a6-6c8ede46d306 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:17.098203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3383502-119f-4417-81a6-6c8ede46d306 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:17.098614 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3383502-119f-4417-81a6-6c8ede46d306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:17.119425 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c3383502-119f-4417-81a6-6c8ede46d306 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:17.120648 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c3383502-119f-4417-81a6-6c8ede46d306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:17.143792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 358/690] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:20:17.145270 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7589e1-53fc-4101-9b7d-9f99fc594917 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:17.192462 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:17.211586 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 333/691] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:20:17.272123 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7589e1-53fc-4101-9b7d-9f99fc594917 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:17.313723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579e1ae-945a-4bf7-a507-584c497490b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:17.314672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579e1ae-945a-4bf7-a507-584c497490b0 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:17.315157 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579e1ae-945a-4bf7-a507-584c497490b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:17.315640 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579e1ae-945a-4bf7-a507-584c497490b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:17.323556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2579e1ae-945a-4bf7-a507-584c497490b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:17.325981 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2579e1ae-945a-4bf7-a507-584c497490b0 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:17.338777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-2579e1ae-945a-4bf7-a507-584c497490b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:17.338777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:17.338777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:17.338777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:17.338777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:17.338777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:17.338777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:17.338777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:17.338777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:17.338777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:17.338777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:17.338777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:17.338777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:17.338777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:17.338777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:17.339706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:17.339706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:17.339706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:17.339706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:17.339706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:17.339706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:17.339706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:17.339706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:17.339706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:17.339706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:17.339706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:17.339706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:17.339706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:17.339706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:17.339706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:17.339706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:17.339706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:17.340687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:17.340687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:17.340687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:17.340687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:17.340687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:17.340687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:17.340687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:17.340687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:17.343201 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 359/692] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:17.348595 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3383502-119f-4417-81a6-6c8ede46d306 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:17.452759 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:17.512177 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3383502-119f-4417-81a6-6c8ede46d306 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:17.562503 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03dd74-f7f8-49bb-bf88-f9d1f140ce6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:17.563486 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03dd74-f7f8-49bb-bf88-f9d1f140ce6f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:17.563996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03dd74-f7f8-49bb-bf88-f9d1f140ce6f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:17.564477 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03dd74-f7f8-49bb-bf88-f9d1f140ce6f None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:17.565390 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d03dd74-f7f8-49bb-bf88-f9d1f140ce6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:17.565909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d03dd74-f7f8-49bb-bf88-f9d1f140ce6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:17.567556 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d03dd74-f7f8-49bb-bf88-f9d1f140ce6f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:17.580973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 334/693] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:17.585576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579e1ae-945a-4bf7-a507-584c497490b0 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:17.589142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03dd74-f7f8-49bb-bf88-f9d1f140ce6f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:17.692452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03dd74-f7f8-49bb-bf88-f9d1f140ce6f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:17.700281 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:17.704851 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579e1ae-945a-4bf7-a507-584c497490b0 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:17.710716 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:17.732284 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8e461-5852-4c63-9465-6ed619b0346c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:17.732895 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8e461-5852-4c63-9465-6ed619b0346c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:17.733170 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8e461-5852-4c63-9465-6ed619b0346c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:17.733437 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8e461-5852-4c63-9465-6ed619b0346c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:17.734210 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8e461-5852-4c63-9465-6ed619b0346c None admin] RBAC: Adding query filter params (role.id=04bc48cc5b6842029b93a845d6744985, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=599129b7d8ff47efb145787414ddb80d) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:17.734445 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8e461-5852-4c63-9465-6ed619b0346c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:17.735881 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8e461-5852-4c63-9465-6ed619b0346c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:17.748218 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 335/694] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:20:17 2022] GET /identity/v3/role_assignments?user.id=599129b7d8ff47efb145787414ddb80d&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=04bc48cc5b6842029b93a845d6744985 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:17.749220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8e461-5852-4c63-9465-6ed619b0346c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:17.760421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fda3f-6f7f-442f-9ba6-62001618d52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:17.761469 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fda3f-6f7f-442f-9ba6-62001618d52d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:17.762584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fda3f-6f7f-442f-9ba6-62001618d52d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:17.763159 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fda3f-6f7f-442f-9ba6-62001618d52d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:17.776064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9fda3f-6f7f-442f-9ba6-62001618d52d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:17.782112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-7a9fda3f-6f7f-442f-9ba6-62001618d52d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:20:17.782112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:17.782112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:17.782112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:17.782112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:17.782112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:17.782112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:17.782112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:17.782112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:17.782112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:17.782112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:17.782112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:17.782112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:17.782112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:17.782112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:17.782112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:17.783040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:17.783040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:17.783040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:17.783040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:17.783040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:17.783040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:17.783040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:17.783040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:17.783040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:17.783040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:17.783040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:17.783040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:17.783040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:17.783040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:17.783040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:17.783040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:17.783861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:17.783861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:17.783861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:17.783861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:17.783861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:17.783861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:17.783861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:17.783861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:17.783861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:17.783861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:17.783861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:20:17.783861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:17.787284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 360/695] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:20:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:17.797728 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fda3f-6f7f-442f-9ba6-62001618d52d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:17.947885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8e461-5852-4c63-9465-6ed619b0346c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:17.975529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fda3f-6f7f-442f-9ba6-62001618d52d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:17.988458 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83ac4f-e9dd-4179-8dba-350f1824fc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:17.989124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83ac4f-e9dd-4179-8dba-350f1824fc4e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:17.989379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83ac4f-e9dd-4179-8dba-350f1824fc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:17.989646 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83ac4f-e9dd-4179-8dba-350f1824fc4e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:17.990229 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83ac4f-e9dd-4179-8dba-350f1824fc4e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:17.991595 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83ac4f-e9dd-4179-8dba-350f1824fc4e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:18.004925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-ca83ac4f-e9dd-4179-8dba-350f1824fc4e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:20:18.004925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:18.004925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:18.004925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:18.004925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:18.004925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:18.004925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:18.004925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:18.004925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:20:18.004925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:18.004925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:18.004925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:18.004925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:18.004925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:18.004925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:18.004925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:18.004925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:18.005996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:18.005996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:18.005996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:18.005996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:18.005996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:18.005996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:18.005996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:18.005996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:18.005996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:18.005996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:18.005996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:18.005996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:18.005996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:18.005996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:18.005996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:18.005996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:18.007872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:18.007872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:18.007872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:18.007872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:18.007872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:18.007872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:18.007872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:18.007872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:18.007872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:18.007872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:18.007872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:18.007872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:18.007872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:20:18.007872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:18.007872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 336/696] 10.222.0.18 () {56 vars in 1129 bytes} [Mon Aug 29 20:20:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:18.012337 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83ac4f-e9dd-4179-8dba-350f1824fc4e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:18.022627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456f616-00fb-431f-bfcd-3a1c6545b9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:18.023590 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456f616-00fb-431f-bfcd-3a1c6545b9f9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:18.024184 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456f616-00fb-431f-bfcd-3a1c6545b9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:18.024676 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456f616-00fb-431f-bfcd-3a1c6545b9f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:18.025584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456f616-00fb-431f-bfcd-3a1c6545b9f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:18.026100 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456f616-00fb-431f-bfcd-3a1c6545b9f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:18.028293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456f616-00fb-431f-bfcd-3a1c6545b9f9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:18.038366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 361/697] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:20:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:18.044922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456f616-00fb-431f-bfcd-3a1c6545b9f9 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:18.064647 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:18.066927 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:18.126353 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83ac4f-e9dd-4179-8dba-350f1824fc4e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:18.165766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456f616-00fb-431f-bfcd-3a1c6545b9f9 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:18.168259 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee30e0d-d61e-4ce1-b78d-1343544a1e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:18.169534 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee30e0d-d61e-4ce1-b78d-1343544a1e2a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:18.169534 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee30e0d-d61e-4ce1-b78d-1343544a1e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:18.169714 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee30e0d-d61e-4ce1-b78d-1343544a1e2a None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:18.170399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee30e0d-d61e-4ce1-b78d-1343544a1e2a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:18.170768 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee30e0d-d61e-4ce1-b78d-1343544a1e2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:18.172254 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee30e0d-d61e-4ce1-b78d-1343544a1e2a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:18.177664 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 337/698] 10.222.0.18 () {56 vars in 1124 bytes} [Mon Aug 29 20:20:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:18.183738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee30e0d-d61e-4ce1-b78d-1343544a1e2a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:18.206040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c92c9-4b64-402a-ab17-88fa0ffe1bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:18.206990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c92c9-4b64-402a-ab17-88fa0ffe1bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:18.207561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c92c9-4b64-402a-ab17-88fa0ffe1bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:18.208070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c92c9-4b64-402a-ab17-88fa0ffe1bc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:18.220731 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c92c9-4b64-402a-ab17-88fa0ffe1bc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:18.222670 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c92c9-4b64-402a-ab17-88fa0ffe1bc7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:18.238987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-183c92c9-4b64-402a-ab17-88fa0ffe1bc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:18.238987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:18.238987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:18.238987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:18.238987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:18.238987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:18.238987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:18.238987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:18.238987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:18.238987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:18.238987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:18.238987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:18.238987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:18.238987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:18.238987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:18.238987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:18.240063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:18.240063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:18.240063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:18.240063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:18.240063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:18.240063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:18.240063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:18.240063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:18.240063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:18.240063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:18.240063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:18.240063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:18.240063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:18.240063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:18.240063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:18.240063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:18.240063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:18.240992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:18.240992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:18.240992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:18.243406 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 362/699] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:18.248776 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c92c9-4b64-402a-ab17-88fa0ffe1bc7 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:18.295835 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee30e0d-d61e-4ce1-b78d-1343544a1e2a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:18.309639 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:18.330776 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:18.338572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e265124-77fb-4a63-89d6-5dbd0d838d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:18.339513 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e265124-77fb-4a63-89d6-5dbd0d838d63 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:18.340739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e265124-77fb-4a63-89d6-5dbd0d838d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:18.341211 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e265124-77fb-4a63-89d6-5dbd0d838d63 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:18.342134 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e265124-77fb-4a63-89d6-5dbd0d838d63 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:18.342651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e265124-77fb-4a63-89d6-5dbd0d838d63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:18.344289 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e265124-77fb-4a63-89d6-5dbd0d838d63 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:18.356469 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 338/700] 10.222.0.18 () {56 vars in 1124 bytes} [Mon Aug 29 20:20:18 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:18.373551 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c92c9-4b64-402a-ab17-88fa0ffe1bc7 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:18.416875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771fc47-50e3-4e13-9943-d55b7f473e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:18.417697 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771fc47-50e3-4e13-9943-d55b7f473e31 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:18.417805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771fc47-50e3-4e13-9943-d55b7f473e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:18.418455 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3771fc47-50e3-4e13-9943-d55b7f473e31 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:18.419901 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3771fc47-50e3-4e13-9943-d55b7f473e31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:18.420264 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3771fc47-50e3-4e13-9943-d55b7f473e31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:18.428382 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3771fc47-50e3-4e13-9943-d55b7f473e31 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:18.452029 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 363/701] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:18.469170 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e265124-77fb-4a63-89d6-5dbd0d838d63 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:18.633721 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e265124-77fb-4a63-89d6-5dbd0d838d63 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:18.671976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc235d9-3ecb-4714-91b9-1855cc133415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:18.672987 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc235d9-3ecb-4714-91b9-1855cc133415 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:18.673556 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc235d9-3ecb-4714-91b9-1855cc133415 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:18.674047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc235d9-3ecb-4714-91b9-1855cc133415 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:18.684355 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc235d9-3ecb-4714-91b9-1855cc133415 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:18.686109 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc235d9-3ecb-4714-91b9-1855cc133415 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:18.699392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-ccc235d9-3ecb-4714-91b9-1855cc133415 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:20:18.699392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:18.699392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:18.699392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:18.699392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:18.699392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:18.699392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:18.699392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:18.699392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:18.699392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:18.699392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:18.699392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:18.699392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:18.699392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:18.699392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:18.699392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:18.700492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:18.700492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:18.700492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:18.700492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:18.700492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:18.700492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:18.700492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:18.700492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:18.700492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:18.700492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:18.700492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:18.700492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:18.700492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:18.700492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:18.700492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:18.700492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:18.700492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:18.700492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:18.701413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:18.701413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:18.701413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:18.701413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:18.701413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:18.701413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:18.701413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:18.701413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:18.701413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:20:18.701413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:18.704007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 339/702] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Aug 29 20:20:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:18.711244 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771fc47-50e3-4e13-9943-d55b7f473e31 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:18.838836 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3771fc47-50e3-4e13-9943-d55b7f473e31 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:18.869028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:18.878864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3660d4e-dc57-47bc-b2ea-93eed5451788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:18.880325 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3660d4e-dc57-47bc-b2ea-93eed5451788 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:18.880835 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3660d4e-dc57-47bc-b2ea-93eed5451788 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:18.881382 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3660d4e-dc57-47bc-b2ea-93eed5451788 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:18.882940 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3660d4e-dc57-47bc-b2ea-93eed5451788 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:18.883606 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3660d4e-dc57-47bc-b2ea-93eed5451788 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:18.885808 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3660d4e-dc57-47bc-b2ea-93eed5451788 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:18.906822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 364/703] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Aug 29 20:20:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:18.927485 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc235d9-3ecb-4714-91b9-1855cc133415 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:19.004309 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:19.136217 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc235d9-3ecb-4714-91b9-1855cc133415 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:19.165193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eec5f7-fa93-4b90-a7fe-84ba72b3ac4e None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:19.165833 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eec5f7-fa93-4b90-a7fe-84ba72b3ac4e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:19.166315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eec5f7-fa93-4b90-a7fe-84ba72b3ac4e None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:19.167832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 365/704] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:19.174332 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d34e4-4ed6-4390-86d6-40f144798914 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:19.174806 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d34e4-4ed6-4390-86d6-40f144798914 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:19.175268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d34e4-4ed6-4390-86d6-40f144798914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:19.186267 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f22bb-16be-4cf8-93e6-fdffab724f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:19.187204 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f22bb-16be-4cf8-93e6-fdffab724f43 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:19.187680 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f22bb-16be-4cf8-93e6-fdffab724f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:19.188199 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f22bb-16be-4cf8-93e6-fdffab724f43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:19.213867 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-3b1d34e4-4ed6-4390-86d6-40f144798914 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:19.216919 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3b1d34e4-4ed6-4390-86d6-40f144798914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:19.320430 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 366/705] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:20:19.322910 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f22bb-16be-4cf8-93e6-fdffab724f43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:19.325374 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f22bb-16be-4cf8-93e6-fdffab724f43 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:19.368153 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:19.393963 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f354fa-9f1d-4fdb-ac25-7c7b6d6dc92e None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:19.394246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f354fa-9f1d-4fdb-ac25-7c7b6d6dc92e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:19.394442 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f354fa-9f1d-4fdb-ac25-7c7b6d6dc92e None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:19.395991 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 367/706] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:20:19.406315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789dca9-79ab-41b0-82ea-c9c8061815f1 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:19.406634 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789dca9-79ab-41b0-82ea-c9c8061815f1 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:19.406877 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789dca9-79ab-41b0-82ea-c9c8061815f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:19.447470 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 340/707] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:20:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:20:19.448739 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d789dca9-79ab-41b0-82ea-c9c8061815f1 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:19.450646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d789dca9-79ab-41b0-82ea-c9c8061815f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:19.453164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f22bb-16be-4cf8-93e6-fdffab724f43 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:19.558604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 368/708] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:20:19.560059 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f22bb-16be-4cf8-93e6-fdffab724f43 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:19.598139 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229d6f2-a64d-4c98-bbb8-200f768163d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:19.599134 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229d6f2-a64d-4c98-bbb8-200f768163d8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:19.599645 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229d6f2-a64d-4c98-bbb8-200f768163d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:19.600161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229d6f2-a64d-4c98-bbb8-200f768163d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:19.611646 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229d6f2-a64d-4c98-bbb8-200f768163d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:19.613664 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229d6f2-a64d-4c98-bbb8-200f768163d8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:19.617543 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:19.631904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-d229d6f2-a64d-4c98-bbb8-200f768163d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:19.631904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:19.631904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:19.631904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:19.631904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:19.631904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:19.631904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:19.631904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:19.631904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:19.631904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:19.631904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:19.631904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:19.631904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:19.631904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:19.631904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:19.631904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:19.632914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:19.632914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:19.632914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:19.632914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:19.632914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:19.632914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:19.632914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:19.632914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:19.632914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:19.632914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:19.632914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:19.632914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:19.632914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:19.632914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:19.632914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:19.632914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:19.632914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:19.633856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:19.633856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:19.633856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:19.636103 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 341/709] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:19.643804 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789dca9-79ab-41b0-82ea-c9c8061815f1 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:19.689768 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229d6f2-a64d-4c98-bbb8-200f768163d8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:19.741156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:19.810420 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229d6f2-a64d-4c98-bbb8-200f768163d8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:19.864090 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789dca9-79ab-41b0-82ea-c9c8061815f1 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:19.869987 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a85a2-0c4e-43d0-bd01-be914ab925f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:19.870658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a85a2-0c4e-43d0-bd01-be914ab925f8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:19.870935 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a85a2-0c4e-43d0-bd01-be914ab925f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:19.871231 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a85a2-0c4e-43d0-bd01-be914ab925f8 None admin] PATH_INFO: `/v3/services/network` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:19.871841 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a85a2-0c4e-43d0-bd01-be914ab925f8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:19.873291 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a85a2-0c4e-43d0-bd01-be914ab925f8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:19.885992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-d96a85a2-0c4e-43d0-bd01-be914ab925f8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:20:19.885992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:19.885992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:19.885992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:19.885992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:19.885992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:19.885992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:19.885992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:19.885992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:20:19.885992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:19.885992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:19.885992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:19.885992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:19.885992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:19.885992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:19.885992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:19.885992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:19.887798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:19.887798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:19.887798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:19.887798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:19.887798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:19.887798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:19.887798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:19.887798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:19.887798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:19.887798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:19.887798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:19.887798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:19.887798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:19.887798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:19.887798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:19.887798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:19.888698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:19.888698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:19.888698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:19.888698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:19.888698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:19.888698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:19.888698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:19.888698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:19.888698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:19.888698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:19.888698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:19.888698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:19.888698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:20:19.888698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:19.890263 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 342/710] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:19 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:19.896994 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a85a2-0c4e-43d0-bd01-be914ab925f8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:19.914249 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088bceb8-5a9f-4292-bff8-4322f75cdae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:19.915053 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bceb8-5a9f-4292-bff8-4322f75cdae6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:19.915339 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bceb8-5a9f-4292-bff8-4322f75cdae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:19.915614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bceb8-5a9f-4292-bff8-4322f75cdae6 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:19.916346 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088bceb8-5a9f-4292-bff8-4322f75cdae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:19.916651 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088bceb8-5a9f-4292-bff8-4322f75cdae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:19.918310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088bceb8-5a9f-4292-bff8-4322f75cdae6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:19.937418 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 369/711] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:19.938723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088bceb8-5a9f-4292-bff8-4322f75cdae6 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:19.984444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:20.037277 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a85a2-0c4e-43d0-bd01-be914ab925f8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:20.057866 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088bceb8-5a9f-4292-bff8-4322f75cdae6 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:20.065661 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:20.080882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a5084-9690-4861-9ab2-e647be8234dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:20.081844 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a5084-9690-4861-9ab2-e647be8234dc None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:20.082453 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a5084-9690-4861-9ab2-e647be8234dc None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:20.082927 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a5084-9690-4861-9ab2-e647be8234dc None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:20.083808 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a5084-9690-4861-9ab2-e647be8234dc None admin] RBAC: Adding query filter params (name=network) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:20.084386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a5084-9690-4861-9ab2-e647be8234dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:20.086180 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a5084-9690-4861-9ab2-e647be8234dc None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:20.091732 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 343/712] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 29 20:20:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:20.092995 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a5084-9690-4861-9ab2-e647be8234dc None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:20.093953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e875e-30fb-4cc5-a518-f800ee9bc8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:20.094598 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e875e-30fb-4cc5-a518-f800ee9bc8ba None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:20.094879 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e875e-30fb-4cc5-a518-f800ee9bc8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:20.095155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e875e-30fb-4cc5-a518-f800ee9bc8ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:20.106870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e875e-30fb-4cc5-a518-f800ee9bc8ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:20.108231 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e875e-30fb-4cc5-a518-f800ee9bc8ba None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:20.123824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-241e875e-30fb-4cc5-a518-f800ee9bc8ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:20.123824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:20.123824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:20.123824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:20.123824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:20.123824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:20.123824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:20.123824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:20.123824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:20.123824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:20.123824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:20.123824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:20.123824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:20.123824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:20.123824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:20.123824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:20.124889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:20.124889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:20.124889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:20.124889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:20.124889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:20.124889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:20.124889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:20.124889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:20.124889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:20.124889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:20.124889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:20.124889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:20.124889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:20.124889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:20.124889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:20.124889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:20.124889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:20.126620 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:20.126620 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:20.126620 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:20.126620 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 370/713] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:20.126620 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e875e-30fb-4cc5-a518-f800ee9bc8ba None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:20.211480 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a5084-9690-4861-9ab2-e647be8234dc None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:20.232674 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e875e-30fb-4cc5-a518-f800ee9bc8ba None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:20.237102 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:20.253625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5627657-e70c-4744-9842-9c364544c7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:20.255224 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5627657-e70c-4744-9842-9c364544c7ef None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:20.256029 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5627657-e70c-4744-9842-9c364544c7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:20.256528 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5627657-e70c-4744-9842-9c364544c7ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:20.265736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5627657-e70c-4744-9842-9c364544c7ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:20.267369 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5627657-e70c-4744-9842-9c364544c7ef None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:20.273468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd165cb1-182d-4889-b0bc-a21a972d3ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:20.274537 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd165cb1-182d-4889-b0bc-a21a972d3ecc None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:20.275130 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd165cb1-182d-4889-b0bc-a21a972d3ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:20.275585 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd165cb1-182d-4889-b0bc-a21a972d3ecc None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:20.276644 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd165cb1-182d-4889-b0bc-a21a972d3ecc None admin] RBAC: Adding query filter params (type=network) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:20.277155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd165cb1-182d-4889-b0bc-a21a972d3ecc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:20.279218 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd165cb1-182d-4889-b0bc-a21a972d3ecc None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:20.280999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-f5627657-e70c-4744-9842-9c364544c7ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:20.280999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:20.280999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:20.280999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:20.280999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:20.280999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:20.280999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:20.280999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:20.280999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:20.280999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:20.280999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:20.280999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:20.280999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:20.280999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:20.280999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:20.281897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:20.281897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:20.281897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:20.281897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:20.281897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:20.281897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:20.281897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:20.281897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:20.281897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:20.281897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:20.281897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:20.281897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:20.281897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:20.281897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:20.281897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:20.281897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:20.281897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:20.282773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:20.282773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:20.282773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:20.282773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:20.282773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:20.282773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:20.282773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:20.282773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:20.287578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 344/714] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:20.289654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 371/715] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 29 20:20:20 2022] GET /identity/v3/services?type=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:20.290917 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd165cb1-182d-4889-b0bc-a21a972d3ecc None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:20.294900 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5627657-e70c-4744-9842-9c364544c7ef None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:20.335966 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:20.501739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5627657-e70c-4744-9842-9c364544c7ef None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:20.504617 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd165cb1-182d-4889-b0bc-a21a972d3ecc None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:20.553224 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625f4d5-d9c8-4a86-b7be-b7e84a22b98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:20.553872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625f4d5-d9c8-4a86-b7be-b7e84a22b98d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:20.554193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625f4d5-d9c8-4a86-b7be-b7e84a22b98d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:20.554520 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625f4d5-d9c8-4a86-b7be-b7e84a22b98d None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:20.555659 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625f4d5-d9c8-4a86-b7be-b7e84a22b98d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:20.556021 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625f4d5-d9c8-4a86-b7be-b7e84a22b98d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:20.558202 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625f4d5-d9c8-4a86-b7be-b7e84a22b98d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:20.561530 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1037f50-4637-48e1-96d1-c646079e5fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:20.562938 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1037f50-4637-48e1-96d1-c646079e5fdf None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:20.563435 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1037f50-4637-48e1-96d1-c646079e5fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:20.563912 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1037f50-4637-48e1-96d1-c646079e5fdf None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:20.564801 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1037f50-4637-48e1-96d1-c646079e5fdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:20.565310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1037f50-4637-48e1-96d1-c646079e5fdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:20.567835 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1037f50-4637-48e1-96d1-c646079e5fdf None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:20.575048 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 345/716] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:20.579547 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625f4d5-d9c8-4a86-b7be-b7e84a22b98d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:20.585191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 372/717] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:20.589368 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1037f50-4637-48e1-96d1-c646079e5fdf None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:20.647226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:20.744434 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:20.753871 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1037f50-4637-48e1-96d1-c646079e5fdf None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:20.776912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625f4d5-d9c8-4a86-b7be-b7e84a22b98d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:20.791255 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8edbf3-0d45-4949-aa36-aa462c175bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:20.791892 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8edbf3-0d45-4949-aa36-aa462c175bc9 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:20.792297 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8edbf3-0d45-4949-aa36-aa462c175bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:20.792575 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8edbf3-0d45-4949-aa36-aa462c175bc9 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:20.793897 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8edbf3-0d45-4949-aa36-aa462c175bc9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:20.795333 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8edbf3-0d45-4949-aa36-aa462c175bc9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:20.817666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee2d69-daca-4db7-a78c-26a0cbc16035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:20.818323 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee2d69-daca-4db7-a78c-26a0cbc16035 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:20.818603 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee2d69-daca-4db7-a78c-26a0cbc16035 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:20.818928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee2d69-daca-4db7-a78c-26a0cbc16035 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:20.819655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee2d69-daca-4db7-a78c-26a0cbc16035 None admin] RBAC: Adding query filter params (role.id=322da7bd64024770a64bdb73e5c54719, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=c0c44754a1c04a2b87081aba6cb509d3) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:20.820040 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee2d69-daca-4db7-a78c-26a0cbc16035 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:20.820484 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 373/718] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 29 20:20:20 2022] POST /identity/v3/users => generated 254 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:20:20.821562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee2d69-daca-4db7-a78c-26a0cbc16035 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:20.825914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ade36-5241-4bc0-86bc-67a87d32a497 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:20.826261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ade36-5241-4bc0-86bc-67a87d32a497 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:20.826541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ade36-5241-4bc0-86bc-67a87d32a497 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:20.828231 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 374/719] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:20.829016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 346/720] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:20:20 2022] GET /identity/v3/role_assignments?user.id=c0c44754a1c04a2b87081aba6cb509d3&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=322da7bd64024770a64bdb73e5c54719 => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:20.834219 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689c2a4-83d4-4c69-8c2e-71b1282d905c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:20.834884 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689c2a4-83d4-4c69-8c2e-71b1282d905c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:20.835367 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689c2a4-83d4-4c69-8c2e-71b1282d905c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:20.877232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-6689c2a4-83d4-4c69-8c2e-71b1282d905c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:20.879054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6689c2a4-83d4-4c69-8c2e-71b1282d905c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:20.997444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c3ec7-5fdc-4c77-9f6e-57770029d935 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:20.998087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c3ec7-5fdc-4c77-9f6e-57770029d935 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:20.998402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 375/721] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:20:20.998882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c3ec7-5fdc-4c77-9f6e-57770029d935 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:21.002593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 347/722] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:20:21.025682 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2d089-66a9-4aeb-b3ff-2e9088c3379b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:21.025962 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2d089-66a9-4aeb-b3ff-2e9088c3379b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:21.026243 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2d089-66a9-4aeb-b3ff-2e9088c3379b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:21.047556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b5d2d089-66a9-4aeb-b3ff-2e9088c3379b None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:21.049143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b5d2d089-66a9-4aeb-b3ff-2e9088c3379b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:21.115345 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:21.123255 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 376/723] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:20:21.307468 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c3ec7-5fdc-4c77-9f6e-57770029d935 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:21.418959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2d089-66a9-4aeb-b3ff-2e9088c3379b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:21.443020 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:21.452122 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c3ec7-5fdc-4c77-9f6e-57770029d935 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:21.497783 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c7d39-a1c6-45ac-80aa-1f15cb6f91ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:21.498426 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c7d39-a1c6-45ac-80aa-1f15cb6f91ef None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:21.498700 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c7d39-a1c6-45ac-80aa-1f15cb6f91ef None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:21.498968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c7d39-a1c6-45ac-80aa-1f15cb6f91ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:21.521699 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c7d39-a1c6-45ac-80aa-1f15cb6f91ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:21.526638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-ea1c7d39-a1c6-45ac-80aa-1f15cb6f91ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:20:21.526638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:21.526638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:21.526638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:21.526638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:21.526638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:21.526638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:21.526638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:21.526638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:21.526638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:21.526638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:21.526638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:21.526638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:21.526638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:21.526638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:21.526638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:21.527643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:21.527643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:21.527643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:21.527643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:21.527643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:21.527643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:21.527643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:21.527643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:21.527643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:21.527643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:21.527643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:21.527643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:21.527643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:21.527643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:21.527643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:21.527643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:21.528530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:21.528530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:21.528530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:21.528530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:21.528530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:21.528530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:21.528530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:21.528530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:21.528530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:21.528530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:21.528530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:20:21.528530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:21.528530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 348/724] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:20:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:21.537170 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c7d39-a1c6-45ac-80aa-1f15cb6f91ef None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:21.633858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2d089-66a9-4aeb-b3ff-2e9088c3379b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:21.703958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de93e1-2fc4-43f8-a932-51c10985e790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:21.703958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de93e1-2fc4-43f8-a932-51c10985e790 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:21.703958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de93e1-2fc4-43f8-a932-51c10985e790 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:21.703958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de93e1-2fc4-43f8-a932-51c10985e790 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:21.704429 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de93e1-2fc4-43f8-a932-51c10985e790 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:21.706001 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de93e1-2fc4-43f8-a932-51c10985e790 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:21.727850 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 377/725] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 29 20:20:21 2022] POST /identity/v3/services => generated 251 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:20:21.744702 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c7d39-a1c6-45ac-80aa-1f15cb6f91ef None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:21.826848 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa30fa8-0d98-4e46-bd6a-9cd935670cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:21.827485 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa30fa8-0d98-4e46-bd6a-9cd935670cff None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:21.827765 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa30fa8-0d98-4e46-bd6a-9cd935670cff None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:21.828071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa30fa8-0d98-4e46-bd6a-9cd935670cff None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:21.828778 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa30fa8-0d98-4e46-bd6a-9cd935670cff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:21.829082 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa30fa8-0d98-4e46-bd6a-9cd935670cff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:21.830486 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa30fa8-0d98-4e46-bd6a-9cd935670cff None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:21.845035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 349/726] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:20:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:21.850834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de93e1-2fc4-43f8-a932-51c10985e790 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:21.949710 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:22.001168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de93e1-2fc4-43f8-a932-51c10985e790 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:22.043736 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02faa3cd-90f0-40e2-b6fb-a6085a307e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:22.044724 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faa3cd-90f0-40e2-b6fb-a6085a307e58 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:22.045164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faa3cd-90f0-40e2-b6fb-a6085a307e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:22.045642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02faa3cd-90f0-40e2-b6fb-a6085a307e58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:22.059198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02faa3cd-90f0-40e2-b6fb-a6085a307e58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:22.062154 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02faa3cd-90f0-40e2-b6fb-a6085a307e58 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:22.083293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-02faa3cd-90f0-40e2-b6fb-a6085a307e58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:22.083293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:22.083293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:22.083293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:22.083293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:22.083293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:22.083293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:22.083293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:22.083293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:22.083293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:22.083293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:22.083293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:22.083293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:22.083293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:22.083293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:22.083293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:22.084390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:22.084390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:22.084390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:22.084390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:22.084390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:22.084390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:22.084390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:22.084390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:22.084390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:22.084390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:22.084390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:22.084390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:22.084390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:22.084390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:22.084390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:22.084390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:22.084390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:22.085300 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:22.085300 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:22.085300 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:22.085476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 378/727] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:22.092877 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa30fa8-0d98-4e46-bd6a-9cd935670cff None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:22.115955 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:22.230006 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa30fa8-0d98-4e46-bd6a-9cd935670cff None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:22.274871 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f591b7c-62fb-45cb-8040-ce371e088714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:22.275487 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f591b7c-62fb-45cb-8040-ce371e088714 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:22.275768 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f591b7c-62fb-45cb-8040-ce371e088714 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:22.276064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f591b7c-62fb-45cb-8040-ce371e088714 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:22.276712 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f591b7c-62fb-45cb-8040-ce371e088714 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:22.277013 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f591b7c-62fb-45cb-8040-ce371e088714 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:22.278910 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f591b7c-62fb-45cb-8040-ce371e088714 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:22.295581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 350/728] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:22.301236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02faa3cd-90f0-40e2-b6fb-a6085a307e58 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:22.494369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02faa3cd-90f0-40e2-b6fb-a6085a307e58 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:22.505962 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:22.534098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629206c8-e6b1-4b9b-8534-dff6a91251c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:22.535125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629206c8-e6b1-4b9b-8534-dff6a91251c5 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:22.535643 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629206c8-e6b1-4b9b-8534-dff6a91251c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:22.536227 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629206c8-e6b1-4b9b-8534-dff6a91251c5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:22.546886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629206c8-e6b1-4b9b-8534-dff6a91251c5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:22.549154 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629206c8-e6b1-4b9b-8534-dff6a91251c5 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:22.561872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-629206c8-e6b1-4b9b-8534-dff6a91251c5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:20:22.561872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:22.561872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:22.561872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:22.561872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:22.561872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:22.561872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:22.561872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:22.561872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:22.561872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:22.561872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:22.561872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:22.561872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:22.561872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:22.561872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:22.561872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:22.563015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:22.563015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:22.563015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:22.563015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:22.563015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:22.563015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:22.563015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:22.563015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:22.563015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:22.563015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:22.563015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:22.563015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:22.563015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:22.563015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:22.563015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:22.563015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:22.563015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:22.563015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:22.563992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:22.563992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:22.563992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:22.563992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:22.563992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:22.563992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:22.563992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:22.563992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:22.563992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:20:22.563992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:22.566613 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 379/729] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:20:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:22.573291 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f591b7c-62fb-45cb-8040-ce371e088714 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:22.717147 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:22.754713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f591b7c-62fb-45cb-8040-ce371e088714 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:22.761497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf91cd5-906d-4f59-b8e0-bb2d8e6a7f69 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:22.761838 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf91cd5-906d-4f59-b8e0-bb2d8e6a7f69 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:22.762180 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf91cd5-906d-4f59-b8e0-bb2d8e6a7f69 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:22.763952 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 380/730] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:22.773103 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ccbb2-bcf5-4e97-acbc-01c66e7470f7 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:22.773396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ccbb2-bcf5-4e97-acbc-01c66e7470f7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:22.773675 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ccbb2-bcf5-4e97-acbc-01c66e7470f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:22.810198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-215ccbb2-bcf5-4e97-acbc-01c66e7470f7 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:22.811392 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-215ccbb2-bcf5-4e97-acbc-01c66e7470f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:22.814460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2f224-b03e-4bbf-ad5c-b3c3467b46cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:22.815082 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2f224-b03e-4bbf-ad5c-b3c3467b46cc None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:22.815358 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2f224-b03e-4bbf-ad5c-b3c3467b46cc None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:22.815624 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2f224-b03e-4bbf-ad5c-b3c3467b46cc None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:22.816385 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d2f224-b03e-4bbf-ad5c-b3c3467b46cc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:22.816687 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d2f224-b03e-4bbf-ad5c-b3c3467b46cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:22.818113 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d2f224-b03e-4bbf-ad5c-b3c3467b46cc None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:22.860806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 351/731] 10.222.0.18 () {56 vars in 1135 bytes} [Mon Aug 29 20:20:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:22.862025 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2f224-b03e-4bbf-ad5c-b3c3467b46cc None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:22.972716 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 381/732] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:22 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 29 20:20:23.025334 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2f224-b03e-4bbf-ad5c-b3c3467b46cc None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:23.061786 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53db1b-eb96-4419-996f-18dccef0b5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:23.062763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53db1b-eb96-4419-996f-18dccef0b5d1 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:23.063236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53db1b-eb96-4419-996f-18dccef0b5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:23.063697 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53db1b-eb96-4419-996f-18dccef0b5d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:23.072354 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:23.134701 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53db1b-eb96-4419-996f-18dccef0b5d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:23.136524 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53db1b-eb96-4419-996f-18dccef0b5d1 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:23.183218 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326c5d2-afb8-4c69-a9f6-885da3fcfe1b None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:23.184099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326c5d2-afb8-4c69-a9f6-885da3fcfe1b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:23.184660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b326c5d2-afb8-4c69-a9f6-885da3fcfe1b None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:23.186660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 382/733] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:20:23.193287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d15aa-1bdf-452a-8d40-e76acb05d201 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:23.193836 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d15aa-1bdf-452a-8d40-e76acb05d201 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:23.194315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d15aa-1bdf-452a-8d40-e76acb05d201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:23.216357 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d64d15aa-1bdf-452a-8d40-e76acb05d201 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:23.217538 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d64d15aa-1bdf-452a-8d40-e76acb05d201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:23.295159 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 352/734] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:20:22 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 29 20:20:23.301337 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53db1b-eb96-4419-996f-18dccef0b5d1 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:23.315609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 383/735] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:23 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 29 20:20:23.344020 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:23.407324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53db1b-eb96-4419-996f-18dccef0b5d1 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:23.438029 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d15aa-1bdf-452a-8d40-e76acb05d201 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:23.444726 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461f360-8f27-4f2e-9cbb-16e6af7ed3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:23.446353 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461f360-8f27-4f2e-9cbb-16e6af7ed3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:23.446640 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461f360-8f27-4f2e-9cbb-16e6af7ed3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:23.446915 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461f360-8f27-4f2e-9cbb-16e6af7ed3f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:23.459204 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461f360-8f27-4f2e-9cbb-16e6af7ed3f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:23.460778 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461f360-8f27-4f2e-9cbb-16e6af7ed3f6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:23.478212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-d461f360-8f27-4f2e-9cbb-16e6af7ed3f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:23.478212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:23.478212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:23.478212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:23.478212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:23.478212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:23.478212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:23.478212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:23.478212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:23.478212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:23.478212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:23.478212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:23.478212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:23.478212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:23.478212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:23.478212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:23.479203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:23.479203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:23.479203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:23.479203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:23.479203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:23.479203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:23.479203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:23.479203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:23.479203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:23.479203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:23.479203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:23.479203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:23.479203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:23.479203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:23.479203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:23.479203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:23.479203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:23.480112 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:23.480112 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:23.480112 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:23.480112 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 353/736] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:23.485725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461f360-8f27-4f2e-9cbb-16e6af7ed3f6 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:23.498149 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:23.575366 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:23.591504 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461f360-8f27-4f2e-9cbb-16e6af7ed3f6 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:23.634413 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d15aa-1bdf-452a-8d40-e76acb05d201 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:23.639844 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270bf11-c287-4a2c-9fa6-2d6db5481f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:23.640509 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270bf11-c287-4a2c-9fa6-2d6db5481f29 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:23.640792 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270bf11-c287-4a2c-9fa6-2d6db5481f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:23.641069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270bf11-c287-4a2c-9fa6-2d6db5481f29 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:23.641971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e270bf11-c287-4a2c-9fa6-2d6db5481f29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:23.642342 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e270bf11-c287-4a2c-9fa6-2d6db5481f29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:23.643737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e270bf11-c287-4a2c-9fa6-2d6db5481f29 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:23.661048 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 354/737] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:23.661757 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270bf11-c287-4a2c-9fa6-2d6db5481f29 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:23.681470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182f2a0-c1aa-4805-b64a-413542bd6aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:23.682175 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182f2a0-c1aa-4805-b64a-413542bd6aac None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:23.682527 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182f2a0-c1aa-4805-b64a-413542bd6aac None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:23.682870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182f2a0-c1aa-4805-b64a-413542bd6aac None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:23.683664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182f2a0-c1aa-4805-b64a-413542bd6aac None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:23.685122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182f2a0-c1aa-4805-b64a-413542bd6aac None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:23.700717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 384/738] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 29 20:20:23 2022] POST /identity/v3/services => generated 240 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:20:23.705019 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182f2a0-c1aa-4805-b64a-413542bd6aac None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:23.804098 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270bf11-c287-4a2c-9fa6-2d6db5481f29 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:23.831737 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:23.864957 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182f2a0-c1aa-4805-b64a-413542bd6aac None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:23.865274 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2a0dc-1633-4d18-b6dc-6e176c8f5962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:23.865963 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2a0dc-1633-4d18-b6dc-6e176c8f5962 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:23.866315 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2a0dc-1633-4d18-b6dc-6e176c8f5962 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:23.866593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2a0dc-1633-4d18-b6dc-6e176c8f5962 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:23.882522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2a0dc-1633-4d18-b6dc-6e176c8f5962 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:23.888419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-c2b2a0dc-1633-4d18-b6dc-6e176c8f5962 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:20:23.888419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:23.888419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:23.888419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:23.888419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:23.888419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:23.888419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:23.888419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:23.888419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:23.888419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:23.888419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:23.888419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:23.888419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:23.888419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:23.888419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:23.888419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:23.889386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:23.889386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:23.889386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:23.889386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:23.889386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:23.889386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:23.889386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:23.889386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:23.889386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:23.889386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:23.889386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:23.889386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:23.889386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:23.889386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:23.889386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:23.889386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:23.890206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:23.890206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:23.890206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:23.890206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:23.890206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:23.890206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:23.890206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:23.890206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:23.890206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:23.890206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:23.890206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:20:23.890206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:23.890206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 355/739] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:20:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:23.891498 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1860a8-57dc-4a5a-b1e2-492bcaaa6eed None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:23.891498 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1860a8-57dc-4a5a-b1e2-492bcaaa6eed None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:23.891697 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1860a8-57dc-4a5a-b1e2-492bcaaa6eed None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:23.893424 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 356/740] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:23.898367 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1860a8-57dc-4a5a-b1e2-492bcaaa6eed None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:23.901373 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:23.915603 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75db3d-b579-42b6-9748-9938aab624da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:23.916286 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75db3d-b579-42b6-9748-9938aab624da None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:23.916566 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75db3d-b579-42b6-9748-9938aab624da None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:23.916842 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75db3d-b579-42b6-9748-9938aab624da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:23.923967 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75db3d-b579-42b6-9748-9938aab624da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:23.925432 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75db3d-b579-42b6-9748-9938aab624da None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:23.937662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-db75db3d-b579-42b6-9748-9938aab624da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:23.937662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:23.937662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:23.937662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:23.937662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:23.937662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:23.937662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:23.937662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:23.937662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:23.937662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:23.937662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:23.937662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:23.937662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:23.937662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:23.937662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:23.938609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:23.938609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:23.938609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:23.938609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:23.938609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:23.938609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:23.938609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:23.938609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:23.938609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:23.938609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:23.938609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:23.938609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:23.938609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:23.938609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:23.938609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:23.938609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:23.938609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:23.939485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:23.939485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:23.939485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:23.939485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:23.939485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:23.939485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:23.939485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:23.939485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:23.939485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 385/741] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:23.940497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d2627-d433-4ece-910b-4ec456870424 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:23.940497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d2627-d433-4ece-910b-4ec456870424 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:23.940497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d2627-d433-4ece-910b-4ec456870424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:23.959541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-020d2627-d433-4ece-910b-4ec456870424 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:23.960862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-020d2627-d433-4ece-910b-4ec456870424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:24.027419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1860a8-57dc-4a5a-b1e2-492bcaaa6eed None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:24.051960 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 386/742] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:24.051960 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d2627-d433-4ece-910b-4ec456870424 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:24.097368 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188875de-dced-433f-b1c6-8e5d18dd5a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:24.098750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188875de-dced-433f-b1c6-8e5d18dd5a34 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:24.099033 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188875de-dced-433f-b1c6-8e5d18dd5a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:24.099309 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188875de-dced-433f-b1c6-8e5d18dd5a34 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:24.101662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188875de-dced-433f-b1c6-8e5d18dd5a34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:24.101987 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188875de-dced-433f-b1c6-8e5d18dd5a34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:24.105461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188875de-dced-433f-b1c6-8e5d18dd5a34 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:24.127009 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 357/743] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:20:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:24.132506 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188875de-dced-433f-b1c6-8e5d18dd5a34 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:24.188977 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:24.219546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d2627-d433-4ece-910b-4ec456870424 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:24.256369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2443f-6530-4e32-b885-a00d01af0c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:24.257351 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2443f-6530-4e32-b885-a00d01af0c8e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:24.257834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2443f-6530-4e32-b885-a00d01af0c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:24.258305 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2443f-6530-4e32-b885-a00d01af0c8e None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:24.259207 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e2443f-6530-4e32-b885-a00d01af0c8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:24.259759 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e2443f-6530-4e32-b885-a00d01af0c8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:24.261476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e2443f-6530-4e32-b885-a00d01af0c8e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:24.264355 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:24.276214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 387/744] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:24.283207 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2443f-6530-4e32-b885-a00d01af0c8e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:24.323538 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188875de-dced-433f-b1c6-8e5d18dd5a34 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:24.395099 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0595343-a2bb-4703-97dd-46cbfb717c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:24.395924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0595343-a2bb-4703-97dd-46cbfb717c6b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:24.396237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0595343-a2bb-4703-97dd-46cbfb717c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:24.396513 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0595343-a2bb-4703-97dd-46cbfb717c6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:24.404064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2443f-6530-4e32-b885-a00d01af0c8e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:24.410664 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0595343-a2bb-4703-97dd-46cbfb717c6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:24.412708 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0595343-a2bb-4703-97dd-46cbfb717c6b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:24.432326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-d0595343-a2bb-4703-97dd-46cbfb717c6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:24.432326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:24.432326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:24.432326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:24.432326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:24.432326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:24.432326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:24.432326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:24.432326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:24.432326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:24.432326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:24.432326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:24.432326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:24.432326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:24.432326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:24.432326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:24.433522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:24.433522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:24.433522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:24.433522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:24.433522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:24.433522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:24.433522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:24.433522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:24.433522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:24.433522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:24.433522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:24.433522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:24.433522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:24.433522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:24.433522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:24.433522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:24.433522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:24.434609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:24.434609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:24.434609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:24.434609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 358/745] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:24.434609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0595343-a2bb-4703-97dd-46cbfb717c6b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:24.445609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b8002-3993-467c-93fa-44a9937bcc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:24.446229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b8002-3993-467c-93fa-44a9937bcc97 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:24.446506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b8002-3993-467c-93fa-44a9937bcc97 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:24.446777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b8002-3993-467c-93fa-44a9937bcc97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:24.448333 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b8002-3993-467c-93fa-44a9937bcc97 None admin] RBAC: Adding query filter params (role.id=322da7bd64024770a64bdb73e5c54719, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=599129b7d8ff47efb145787414ddb80d) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:24.448333 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b8002-3993-467c-93fa-44a9937bcc97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:24.450773 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b8002-3993-467c-93fa-44a9937bcc97 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:24.455076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:24.468164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 388/746] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:20:24 2022] GET /identity/v3/role_assignments?user.id=599129b7d8ff47efb145787414ddb80d&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=322da7bd64024770a64bdb73e5c54719 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:24.470005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe9042-98ec-4955-8512-8bdd00a63a18 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:24.470297 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe9042-98ec-4955-8512-8bdd00a63a18 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:24.470577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe9042-98ec-4955-8512-8bdd00a63a18 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:24.473234 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 389/747] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:24.474114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe9042-98ec-4955-8512-8bdd00a63a18 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:24.569656 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0595343-a2bb-4703-97dd-46cbfb717c6b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:24.593583 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:24.620433 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c416255-a7c5-4507-866f-d45adeda6ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:24.621590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c416255-a7c5-4507-866f-d45adeda6ebf None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:24.622024 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c416255-a7c5-4507-866f-d45adeda6ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:24.622662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c416255-a7c5-4507-866f-d45adeda6ebf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:24.633669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c416255-a7c5-4507-866f-d45adeda6ebf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:24.639311 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe9042-98ec-4955-8512-8bdd00a63a18 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:24.639554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-7c416255-a7c5-4507-866f-d45adeda6ebf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:24.639554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:24.639554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:24.639554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:24.639554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:24.639554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:24.639554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:24.639554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:24.639554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:24.639554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:24.639554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:24.639554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:24.639554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:24.639554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:24.639554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:24.639554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:24.641770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:24.641770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:24.641770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:24.641770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:24.641770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:24.641770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:24.641770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:24.641770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:24.641770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:24.641770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:24.641770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:24.641770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:24.641770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:24.641770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:24.641770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:24.641770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:24.642853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:24.642853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:24.642853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:24.642853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:24.642853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:24.642853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:24.642853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:24.642853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:24.642853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:24.642853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:24.642853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:24.642853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:24.642853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 359/748] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:20:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:24.642853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b314535-9861-4c29-a517-a985f320f1d2 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:24.643840 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b314535-9861-4c29-a517-a985f320f1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:24.643840 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b314535-9861-4c29-a517-a985f320f1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:24.674153 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-8b314535-9861-4c29-a517-a985f320f1d2 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:24.675796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8b314535-9861-4c29-a517-a985f320f1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:24.691870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f394e-2246-4010-ad05-293bfa7f18f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:24.692567 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f394e-2246-4010-ad05-293bfa7f18f5 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:24.692844 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f394e-2246-4010-ad05-293bfa7f18f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:24.693162 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f394e-2246-4010-ad05-293bfa7f18f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:24.693830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f394e-2246-4010-ad05-293bfa7f18f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:24.694135 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f394e-2246-4010-ad05-293bfa7f18f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:24.695500 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f394e-2246-4010-ad05-293bfa7f18f5 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:24.715568 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 390/749] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:24.716572 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f394e-2246-4010-ad05-293bfa7f18f5 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:24.763478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 360/750] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:24.764614 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b314535-9861-4c29-a517-a985f320f1d2 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:24.829342 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:24.890081 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b314535-9861-4c29-a517-a985f320f1d2 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:24.898503 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:24.932210 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5353a-f3f3-4b83-87bb-9acca9b89605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:24.933076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5353a-f3f3-4b83-87bb-9acca9b89605 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:24.933505 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5353a-f3f3-4b83-87bb-9acca9b89605 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:24.933972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5353a-f3f3-4b83-87bb-9acca9b89605 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:24.942529 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5353a-f3f3-4b83-87bb-9acca9b89605 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:24.944449 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5353a-f3f3-4b83-87bb-9acca9b89605 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:24.967392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-3dc5353a-f3f3-4b83-87bb-9acca9b89605 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:20:24.967392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:24.967392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:24.967392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:24.967392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:24.967392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:24.967392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:24.967392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:24.967392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:24.967392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:24.967392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:24.967392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:24.967392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:24.967392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:24.967392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:24.967392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:24.968740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:24.968740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:24.968740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:24.968740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:24.968740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:24.968740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:24.968740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:24.968740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:24.968740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:24.968740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:24.968740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:24.968740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:24.968740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:24.968740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:24.968740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:24.968740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:24.968740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:24.968740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:24.970193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:24.970193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:24.970193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:24.970193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:24.970193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:24.970193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:24.970193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:24.970193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:24.970193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:20:24.970193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:24.970193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 361/751] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Aug 29 20:20:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:24.972086 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5353a-f3f3-4b83-87bb-9acca9b89605 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:24.974093 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f394e-2246-4010-ad05-293bfa7f18f5 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:25.052184 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e2ad0f-0021-499e-85b1-21a88d171973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:25.052833 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2ad0f-0021-499e-85b1-21a88d171973 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:25.053085 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2ad0f-0021-499e-85b1-21a88d171973 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:25.053326 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2ad0f-0021-499e-85b1-21a88d171973 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:25.054011 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e2ad0f-0021-499e-85b1-21a88d171973 None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:25.054317 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e2ad0f-0021-499e-85b1-21a88d171973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:25.055698 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e2ad0f-0021-499e-85b1-21a88d171973 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:25.065341 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 391/752] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:20:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:25.070685 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e2ad0f-0021-499e-85b1-21a88d171973 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:25.116460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5353a-f3f3-4b83-87bb-9acca9b89605 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:25.158417 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a846d-83fb-466d-aad5-306132aac6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:25.159028 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a846d-83fb-466d-aad5-306132aac6cd None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:25.159380 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a846d-83fb-466d-aad5-306132aac6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:25.159652 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a846d-83fb-466d-aad5-306132aac6cd None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:25.160409 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a846d-83fb-466d-aad5-306132aac6cd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:25.160788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a846d-83fb-466d-aad5-306132aac6cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:25.162217 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a846d-83fb-466d-aad5-306132aac6cd None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:25.183099 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 362/753] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Aug 29 20:20:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:25.183099 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a846d-83fb-466d-aad5-306132aac6cd None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:25.219490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e2ad0f-0021-499e-85b1-21a88d171973 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:25.225282 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:25.251300 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:25.269244 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d4b88-1455-4335-9587-a5bd086eefe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:25.271020 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d4b88-1455-4335-9587-a5bd086eefe4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:25.271802 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d4b88-1455-4335-9587-a5bd086eefe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:25.272277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d4b88-1455-4335-9587-a5bd086eefe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:25.286218 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d4b88-1455-4335-9587-a5bd086eefe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:25.291567 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d4b88-1455-4335-9587-a5bd086eefe4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:25.316013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-350d4b88-1455-4335-9587-a5bd086eefe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:25.316013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:25.316013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:25.316013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:25.316013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:25.316013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:25.316013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:25.316013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:25.316013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:25.316013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:25.316013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:25.316013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:25.316013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:25.316013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:25.316013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:25.316013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:25.317171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:25.317171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:25.317171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:25.317171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:25.317171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:25.317171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:25.317171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:25.317171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:25.317171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:25.317171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:25.317171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:25.317171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:25.317171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:25.317171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:25.317171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:25.317171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:25.317171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:25.318250 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:25.318250 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:25.318250 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:25.320585 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 392/754] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:25.321621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d4b88-1455-4335-9587-a5bd086eefe4 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:25.326900 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a846d-83fb-466d-aad5-306132aac6cd None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:25.369909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e3195-d68c-44ee-8a39-ba9b5f5109ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:25.370745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e3195-d68c-44ee-8a39-ba9b5f5109ef None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:25.371807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e3195-d68c-44ee-8a39-ba9b5f5109ef None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:25.372208 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e3195-d68c-44ee-8a39-ba9b5f5109ef None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:25.372940 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e3195-d68c-44ee-8a39-ba9b5f5109ef None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:25.374417 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e3195-d68c-44ee-8a39-ba9b5f5109ef None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:25.384430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-f01e3195-d68c-44ee-8a39-ba9b5f5109ef None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:20:25.384430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:25.384430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:25.384430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:25.384430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:25.384430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:25.384430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:25.384430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:25.384430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:20:25.384430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:25.384430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:25.384430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:25.384430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:25.384430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:25.384430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:25.384430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:25.384430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:25.385717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:25.385717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:25.385717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:25.385717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:25.385717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:25.385717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:25.385717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:25.385717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:25.385717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:25.385717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:25.385717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:25.385717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:25.385717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:25.385717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:25.385717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:25.385717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:25.386748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:25.386748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:25.386748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:25.386748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:25.386748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:25.386748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:25.386748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:25.386748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:25.386748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:25.386748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:25.386748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:25.386748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:25.386748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:20:25.386748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:25.389740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 363/755] 10.222.0.18 () {56 vars in 1129 bytes} [Mon Aug 29 20:20:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:25.391096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e3195-d68c-44ee-8a39-ba9b5f5109ef None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:25.452048 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d4b88-1455-4335-9587-a5bd086eefe4 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:25.494411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9423e-ccce-44cc-ada9-22d8af324d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:25.495401 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9423e-ccce-44cc-ada9-22d8af324d3a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:25.495790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9423e-ccce-44cc-ada9-22d8af324d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:25.496857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9423e-ccce-44cc-ada9-22d8af324d3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:25.510987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9423e-ccce-44cc-ada9-22d8af324d3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:25.514030 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9423e-ccce-44cc-ada9-22d8af324d3a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:25.516365 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:25.531106 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e3195-d68c-44ee-8a39-ba9b5f5109ef None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:25.535354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-65c9423e-ccce-44cc-ada9-22d8af324d3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:25.535354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:25.535354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:25.535354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:25.535354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:25.535354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:25.535354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:25.535354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:25.535354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:25.535354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:25.535354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:25.535354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:25.535354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:25.535354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:25.535354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:25.535354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:25.536329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:25.536329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:25.536329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:25.536329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:25.536329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:25.536329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:25.536329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:25.536329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:25.536329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:25.536329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:25.536329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:25.536329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:25.536329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:25.536329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:25.536329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:25.536329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:25.536329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:25.537277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:25.537277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:25.537277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:25.540972 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 393/756] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:25.542091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9423e-ccce-44cc-ada9-22d8af324d3a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:25.576865 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:25.583339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d510e8d-93d1-4941-b7b9-ab6db142b9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:25.584331 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d510e8d-93d1-4941-b7b9-ab6db142b9b4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:25.584686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d510e8d-93d1-4941-b7b9-ab6db142b9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:25.585031 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d510e8d-93d1-4941-b7b9-ab6db142b9b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:25.585759 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d510e8d-93d1-4941-b7b9-ab6db142b9b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:25.586133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d510e8d-93d1-4941-b7b9-ab6db142b9b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:25.587895 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d510e8d-93d1-4941-b7b9-ab6db142b9b4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:25.611648 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 364/757] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:25.611648 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d510e8d-93d1-4941-b7b9-ab6db142b9b4 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:25.676954 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9423e-ccce-44cc-ada9-22d8af324d3a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:25.725344 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b494f-ebb0-46c1-8fbe-3bac2d4c62ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:25.725756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b494f-ebb0-46c1-8fbe-3bac2d4c62ce None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:25.726047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b494f-ebb0-46c1-8fbe-3bac2d4c62ce None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:25.726355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b494f-ebb0-46c1-8fbe-3bac2d4c62ce None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:25.727001 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b494f-ebb0-46c1-8fbe-3bac2d4c62ce None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:25.727315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b494f-ebb0-46c1-8fbe-3bac2d4c62ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:25.728878 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b494f-ebb0-46c1-8fbe-3bac2d4c62ce None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:25.735338 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 394/758] 10.222.0.18 () {56 vars in 1124 bytes} [Mon Aug 29 20:20:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:25.736549 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b494f-ebb0-46c1-8fbe-3bac2d4c62ce None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:25.763773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d510e8d-93d1-4941-b7b9-ab6db142b9b4 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:25.801612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2c9ce-540b-470b-a1c5-e85cfd49cdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:25.802561 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2c9ce-540b-470b-a1c5-e85cfd49cdd9 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:25.803032 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2c9ce-540b-470b-a1c5-e85cfd49cdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:25.803521 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff2c9ce-540b-470b-a1c5-e85cfd49cdd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:25.804390 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2c9ce-540b-470b-a1c5-e85cfd49cdd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:25.804913 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2c9ce-540b-470b-a1c5-e85cfd49cdd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:25.806464 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff2c9ce-540b-470b-a1c5-e85cfd49cdd9 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:25.819624 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 365/759] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:25.820832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2c9ce-540b-470b-a1c5-e85cfd49cdd9 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:25.832439 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:25.842805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:25.854349 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b494f-ebb0-46c1-8fbe-3bac2d4c62ce None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:25.892040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29200b-92a2-43b5-bdec-d6ce75a56367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:25.892689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29200b-92a2-43b5-bdec-d6ce75a56367 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:25.892974 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29200b-92a2-43b5-bdec-d6ce75a56367 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:25.893254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29200b-92a2-43b5-bdec-d6ce75a56367 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:25.902252 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef29200b-92a2-43b5-bdec-d6ce75a56367 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:25.903860 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef29200b-92a2-43b5-bdec-d6ce75a56367 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:25.916389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-ef29200b-92a2-43b5-bdec-d6ce75a56367 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:20:25.916389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:25.916389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:25.916389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:25.916389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:25.916389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:25.916389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:25.916389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:25.916389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:25.916389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:25.916389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:25.916389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:25.916389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:25.916389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:25.916389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:25.916389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:25.917479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:25.917479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:25.917479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:25.917479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:25.917479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:25.917479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:25.917479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:25.917479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:25.917479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:25.917479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:25.917479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:25.917479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:25.917479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:25.917479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:25.917479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:25.917479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:25.917479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:25.917479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:25.919117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:25.919117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:25.919117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:25.919117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:25.919117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:25.919117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:25.919117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:25.919117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:25.919117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:20:25.919117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:25.922876 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 395/760] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Aug 29 20:20:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:25.923832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29200b-92a2-43b5-bdec-d6ce75a56367 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:25.944479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff2c9ce-540b-470b-a1c5-e85cfd49cdd9 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:25.984970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940324f-9be7-42d6-8848-a6a80e8a8fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:25.985499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940324f-9be7-42d6-8848-a6a80e8a8fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:25.985791 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940324f-9be7-42d6-8848-a6a80e8a8fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:25.986123 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940324f-9be7-42d6-8848-a6a80e8a8fe3 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:25.986775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940324f-9be7-42d6-8848-a6a80e8a8fe3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:25.987078 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940324f-9be7-42d6-8848-a6a80e8a8fe3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:25.988490 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940324f-9be7-42d6-8848-a6a80e8a8fe3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:25.993719 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 366/761] 10.222.0.18 () {56 vars in 1124 bytes} [Mon Aug 29 20:20:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:25.994544 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940324f-9be7-42d6-8848-a6a80e8a8fe3 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:26.045881 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29200b-92a2-43b5-bdec-d6ce75a56367 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:26.077151 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:26.085289 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5f246f-c3b5-44c5-b680-99a5273aba01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:26.085948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f246f-c3b5-44c5-b680-99a5273aba01 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:26.086209 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f246f-c3b5-44c5-b680-99a5273aba01 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:26.086484 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f246f-c3b5-44c5-b680-99a5273aba01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:26.094467 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5f246f-c3b5-44c5-b680-99a5273aba01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:26.095923 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5f246f-c3b5-44c5-b680-99a5273aba01 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:26.108477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-2b5f246f-c3b5-44c5-b680-99a5273aba01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:26.108477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:26.108477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:26.108477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:26.108477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:26.108477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:26.108477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:26.108477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:26.108477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:26.108477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:26.108477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:26.108477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:26.108477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:26.108477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:26.108477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:26.109405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:26.109405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:26.109405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:26.109405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:26.109405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:26.109405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:26.109405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:26.109405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:26.109405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:26.109405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:26.109405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:26.109405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:26.109405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:26.109405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:26.109405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:26.109405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:26.109405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:26.110298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:26.110298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:26.110298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:26.110298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:26.110298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:26.110298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:26.110298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:26.110298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:26.110298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 396/762] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:26.111359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e0e36-6cdf-4fbe-bfce-a43c6708524f None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:26.111359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e0e36-6cdf-4fbe-bfce-a43c6708524f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:26.111526 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e0e36-6cdf-4fbe-bfce-a43c6708524f None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:26.112813 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 397/763] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:26.117621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e0e36-6cdf-4fbe-bfce-a43c6708524f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:26.123782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940324f-9be7-42d6-8848-a6a80e8a8fe3 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:26.131005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:26.165960 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b155f17-ce93-4e37-8fa1-441461f93d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:26.166846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b155f17-ce93-4e37-8fa1-441461f93d53 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:26.167225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b155f17-ce93-4e37-8fa1-441461f93d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:26.167571 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b155f17-ce93-4e37-8fa1-441461f93d53 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:26.168914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b155f17-ce93-4e37-8fa1-441461f93d53 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:26.169241 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b155f17-ce93-4e37-8fa1-441461f93d53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:26.170831 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b155f17-ce93-4e37-8fa1-441461f93d53 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:26.195491 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 367/764] 10.222.0.18 () {56 vars in 1149 bytes} [Mon Aug 29 20:20:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:26.195491 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b155f17-ce93-4e37-8fa1-441461f93d53 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:26.236058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e0e36-6cdf-4fbe-bfce-a43c6708524f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:26.278573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42476b-e530-4965-958c-4b546bf0e959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:26.279195 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42476b-e530-4965-958c-4b546bf0e959 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:26.279475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42476b-e530-4965-958c-4b546bf0e959 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:26.279746 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42476b-e530-4965-958c-4b546bf0e959 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:26.280505 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa42476b-e530-4965-958c-4b546bf0e959 None admin] RBAC: Adding query filter params (interface=public, service_id=baf026b6ce8748a9b1565eef0f2e1b3b, region_id=RegionOne) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:26.280817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa42476b-e530-4965-958c-4b546bf0e959 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:26.282199 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa42476b-e530-4965-958c-4b546bf0e959 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:26.298624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 398/765] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Aug 29 20:20:26 2022] GET /identity/v3/endpoints?service_id=baf026b6ce8748a9b1565eef0f2e1b3b&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:26.298624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1e2c5-8f36-4fe8-af24-2ed636ddf16c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:26.298624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1e2c5-8f36-4fe8-af24-2ed636ddf16c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:26.298624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1e2c5-8f36-4fe8-af24-2ed636ddf16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:26.316934 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-bfa1e2c5-8f36-4fe8-af24-2ed636ddf16c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:26.318182 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-bfa1e2c5-8f36-4fe8-af24-2ed636ddf16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:26.322947 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b155f17-ce93-4e37-8fa1-441461f93d53 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:26.359401 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:26.369923 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155931cd-c70d-4112-b7cd-3065ddb1ac36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:26.370889 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155931cd-c70d-4112-b7cd-3065ddb1ac36 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:26.371387 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155931cd-c70d-4112-b7cd-3065ddb1ac36 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:26.371849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155931cd-c70d-4112-b7cd-3065ddb1ac36 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:26.372945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155931cd-c70d-4112-b7cd-3065ddb1ac36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:26.373258 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155931cd-c70d-4112-b7cd-3065ddb1ac36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:26.374867 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155931cd-c70d-4112-b7cd-3065ddb1ac36 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:26.387633 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 368/766] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:26.388594 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155931cd-c70d-4112-b7cd-3065ddb1ac36 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:26.406915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 399/767] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:26.408137 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1e2c5-8f36-4fe8-af24-2ed636ddf16c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:26.417564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:26.526342 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155931cd-c70d-4112-b7cd-3065ddb1ac36 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:26.555245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1e2c5-8f36-4fe8-af24-2ed636ddf16c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:26.572704 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e705eb-6bb1-4935-b6bc-a2f809dce4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:26.574950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e705eb-6bb1-4935-b6bc-a2f809dce4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:26.575790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e705eb-6bb1-4935-b6bc-a2f809dce4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:26.576323 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e705eb-6bb1-4935-b6bc-a2f809dce4f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:26.607984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423c957-1dfb-403d-b6d5-e34b8c9f7563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:26.608641 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423c957-1dfb-403d-b6d5-e34b8c9f7563 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:26.608924 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423c957-1dfb-403d-b6d5-e34b8c9f7563 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:26.609216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423c957-1dfb-403d-b6d5-e34b8c9f7563 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:26.609814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423c957-1dfb-403d-b6d5-e34b8c9f7563 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:26.611539 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423c957-1dfb-403d-b6d5-e34b8c9f7563 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:26.625347 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 400/768] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Aug 29 20:20:26 2022] GET /identity/v3/services => generated 772 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:26.626363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423c957-1dfb-403d-b6d5-e34b8c9f7563 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:26.662181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:26.678894 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e705eb-6bb1-4935-b6bc-a2f809dce4f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:26.680783 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e705eb-6bb1-4935-b6bc-a2f809dce4f8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:26.721815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:26.762701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423c957-1dfb-403d-b6d5-e34b8c9f7563 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:26.806249 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f34533c-651d-4acd-b728-b210ce631ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:26.806952 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34533c-651d-4acd-b728-b210ce631ec1 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:26.807078 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34533c-651d-4acd-b728-b210ce631ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:26.807295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34533c-651d-4acd-b728-b210ce631ec1 None admin] PATH_INFO: `/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/c0c44754a1c04a2b87081aba6cb509d3/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:26.809869 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f34533c-651d-4acd-b728-b210ce631ec1 None admin] RBAC: Authorizing `identity:create_grant(project_id=90b25dc95d5a4a068994bdd801d6b8ea, user_id=c0c44754a1c04a2b87081aba6cb509d3, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:26.811330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f34533c-651d-4acd-b728-b210ce631ec1 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:26.817127 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 369/769] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:20:26 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:20:26.818448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870cc24-2440-4608-9c69-549f0df7815e None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:26.818738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870cc24-2440-4608-9c69-549f0df7815e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:26.819017 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870cc24-2440-4608-9c69-549f0df7815e None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:26.820596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 370/770] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:26.820956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870cc24-2440-4608-9c69-549f0df7815e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:26.835168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 401/771] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:20:26 2022] PUT /identity/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/c0c44754a1c04a2b87081aba6cb509d3/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:20:26.836505 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14519241-a1b2-426e-96b3-e791a4b153af None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:26.836808 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14519241-a1b2-426e-96b3-e791a4b153af None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:26.837082 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14519241-a1b2-426e-96b3-e791a4b153af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:26.857962 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-14519241-a1b2-426e-96b3-e791a4b153af None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:26.859258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-14519241-a1b2-426e-96b3-e791a4b153af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:26.958264 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 402/772] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:26.959277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14519241-a1b2-426e-96b3-e791a4b153af None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:26.959701 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:26.993836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870cc24-2440-4608-9c69-549f0df7815e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:27.026198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:27.067392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fdfcd-e1f6-409b-8453-b9d43b467eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:27.068041 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fdfcd-e1f6-409b-8453-b9d43b467eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:27.068322 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fdfcd-e1f6-409b-8453-b9d43b467eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:27.068593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fdfcd-e1f6-409b-8453-b9d43b467eb2 None admin] PATH_INFO: `/v3/services/network` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:27.069235 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763fdfcd-e1f6-409b-8453-b9d43b467eb2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:27.070777 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763fdfcd-e1f6-409b-8453-b9d43b467eb2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:27.077465 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14519241-a1b2-426e-96b3-e791a4b153af None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:27.086100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-763fdfcd-e1f6-409b-8453-b9d43b467eb2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:20:27.086100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:27.086100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:27.086100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:27.086100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:27.086100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:27.086100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:27.086100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:27.086100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:20:27.086100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:27.086100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:27.086100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:27.086100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:27.086100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:27.086100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:27.086100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:27.086100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:27.087357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:27.087357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:27.087357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:27.087357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:27.087357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:27.087357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:27.087357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:27.087357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:27.087357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:27.087357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:27.087357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:27.087357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:27.087357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:27.087357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:27.087357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:27.087357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:27.088362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:27.088362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:27.088362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:27.088362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:27.088362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:27.088362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:27.088362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:27.088362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:27.088362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:27.088362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:27.088362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:27.088362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:27.088362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:20:27.088362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:27.089209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 371/773] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:26 2022] GET /identity/v3/services/network => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:27.095428 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fdfcd-e1f6-409b-8453-b9d43b467eb2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:27.120464 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb698dc-a6e2-4dc2-9d71-515eade2aa78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:27.122367 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb698dc-a6e2-4dc2-9d71-515eade2aa78 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:27.122811 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb698dc-a6e2-4dc2-9d71-515eade2aa78 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:27.123255 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb698dc-a6e2-4dc2-9d71-515eade2aa78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:27.137375 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb698dc-a6e2-4dc2-9d71-515eade2aa78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:27.139219 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb698dc-a6e2-4dc2-9d71-515eade2aa78 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:27.160742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-3cb698dc-a6e2-4dc2-9d71-515eade2aa78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:27.160742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:27.160742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:27.160742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:27.160742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:27.160742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:27.160742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:27.160742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:27.160742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:27.160742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:27.160742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:27.160742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:27.160742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:27.160742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:27.160742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:27.160742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:27.161815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:27.161815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:27.161815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:27.161815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:27.161815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:27.161815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:27.161815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:27.161815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:27.161815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:27.161815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:27.161815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:27.161815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:27.161815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:27.161815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:27.161815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:27.161815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:27.161815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:27.162722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:27.162722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:27.162722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:27.162722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 403/774] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:27.170628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb698dc-a6e2-4dc2-9d71-515eade2aa78 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:27.294022 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fdfcd-e1f6-409b-8453-b9d43b467eb2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:27.328496 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb698dc-a6e2-4dc2-9d71-515eade2aa78 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:27.338557 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e513a-6541-4a4d-a05d-1f927701fd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:27.338809 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e513a-6541-4a4d-a05d-1f927701fd4e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:27.339399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e513a-6541-4a4d-a05d-1f927701fd4e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:27.339399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e513a-6541-4a4d-a05d-1f927701fd4e None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:27.340059 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e513a-6541-4a4d-a05d-1f927701fd4e None admin] RBAC: Adding query filter params (name=network) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:27.340381 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e513a-6541-4a4d-a05d-1f927701fd4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:27.342653 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e513a-6541-4a4d-a05d-1f927701fd4e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:27.348586 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 372/775] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 29 20:20:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:27.349451 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e513a-6541-4a4d-a05d-1f927701fd4e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:27.352421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:27.380026 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:27.383408 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436cd7e0-6d82-43a4-ae3f-1dae86622e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:27.384384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cd7e0-6d82-43a4-ae3f-1dae86622e78 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:27.384852 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cd7e0-6d82-43a4-ae3f-1dae86622e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:27.385595 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cd7e0-6d82-43a4-ae3f-1dae86622e78 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:27.387038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436cd7e0-6d82-43a4-ae3f-1dae86622e78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:27.387779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436cd7e0-6d82-43a4-ae3f-1dae86622e78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:27.390440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436cd7e0-6d82-43a4-ae3f-1dae86622e78 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:27.408311 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 404/776] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:27.409619 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436cd7e0-6d82-43a4-ae3f-1dae86622e78 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:27.506776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e513a-6541-4a4d-a05d-1f927701fd4e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:27.530508 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436cd7e0-6d82-43a4-ae3f-1dae86622e78 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:27.553795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51285612-5487-4bc9-a60a-7d0067889057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:27.554414 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51285612-5487-4bc9-a60a-7d0067889057 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:27.554691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51285612-5487-4bc9-a60a-7d0067889057 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:27.554961 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51285612-5487-4bc9-a60a-7d0067889057 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:27.567034 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51285612-5487-4bc9-a60a-7d0067889057 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:27.573590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-51285612-5487-4bc9-a60a-7d0067889057 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:20:27.573590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:27.573590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:27.573590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:27.573590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:27.573590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:27.573590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:27.573590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:27.573590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:27.573590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:27.573590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:27.573590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:27.573590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:27.573590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:27.573590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:27.573590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:27.574807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:27.574807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:27.574807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:27.574807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:27.574807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:27.574807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:27.574807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:27.574807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:27.574807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:27.574807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:27.574807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:27.574807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:27.574807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:27.574807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:27.574807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:27.574807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:27.575644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:27.575644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:27.575644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:27.575644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:27.575644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:27.575644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:27.575644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:27.575644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:27.575644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:27.575644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:27.575644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:20:27.575644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:27.576344 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1f619-a33a-4e78-9d5d-2317968370a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:27.576344 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1f619-a33a-4e78-9d5d-2317968370a3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:27.576344 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1f619-a33a-4e78-9d5d-2317968370a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:27.576344 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1f619-a33a-4e78-9d5d-2317968370a3 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:27.577139 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1f619-a33a-4e78-9d5d-2317968370a3 None admin] RBAC: Adding query filter params (type=network) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:27.577139 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1f619-a33a-4e78-9d5d-2317968370a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:27.578612 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1f619-a33a-4e78-9d5d-2317968370a3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:27.585114 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 373/777] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:20:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:27.585114 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51285612-5487-4bc9-a60a-7d0067889057 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:27.586295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 405/778] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 29 20:20:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:27.587034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1f619-a33a-4e78-9d5d-2317968370a3 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:27.647740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:27.692343 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:27.712649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51285612-5487-4bc9-a60a-7d0067889057 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:27.738346 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1f619-a33a-4e78-9d5d-2317968370a3 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:27.756045 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3aa22-7b13-4acc-ac38-8004ce6e972d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:27.757156 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3aa22-7b13-4acc-ac38-8004ce6e972d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:27.757500 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3aa22-7b13-4acc-ac38-8004ce6e972d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:27.757861 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3aa22-7b13-4acc-ac38-8004ce6e972d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:27.768686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3aa22-7b13-4acc-ac38-8004ce6e972d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:27.772243 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3aa22-7b13-4acc-ac38-8004ce6e972d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:27.787248 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1b32b-4af2-4ce1-9d40-25a3d7236a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:27.787855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-abd3aa22-7b13-4acc-ac38-8004ce6e972d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:27.787855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:27.787855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:27.787855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:27.787855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:27.787855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:27.787855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:27.787855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:27.787855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:27.787855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:27.787855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:27.787855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:27.787855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:27.787855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:27.787855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:27.788879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:27.788879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:27.788879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:27.788879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:27.788879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:27.788879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:27.788879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:27.788879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:27.788879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:27.788879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:27.788879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:27.788879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:27.788879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:27.788879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:27.788879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:27.788879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:27.788879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:27.790161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:27.790161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:27.790161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:27.790161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:27.790161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:27.790161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:27.790161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:27.790161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:27.790579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1b32b-4af2-4ce1-9d40-25a3d7236a20 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:27.790579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1b32b-4af2-4ce1-9d40-25a3d7236a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:27.790579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1b32b-4af2-4ce1-9d40-25a3d7236a20 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:27.790579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1b32b-4af2-4ce1-9d40-25a3d7236a20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:27.790579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1b32b-4af2-4ce1-9d40-25a3d7236a20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:27.791019 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1b32b-4af2-4ce1-9d40-25a3d7236a20 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:27.792187 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 374/779] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:27.793954 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3aa22-7b13-4acc-ac38-8004ce6e972d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:27.801793 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 406/780] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:20:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:27.802764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1b32b-4af2-4ce1-9d40-25a3d7236a20 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:27.898422 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3aa22-7b13-4acc-ac38-8004ce6e972d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:27.915018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:27.923626 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1b32b-4af2-4ce1-9d40-25a3d7236a20 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:27.935503 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0cd33b-b25a-4d6d-9c7b-c15f8b3ca196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:27.936807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cd33b-b25a-4d6d-9c7b-c15f8b3ca196 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:27.937283 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cd33b-b25a-4d6d-9c7b-c15f8b3ca196 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:27.937874 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cd33b-b25a-4d6d-9c7b-c15f8b3ca196 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:27.939251 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0cd33b-b25a-4d6d-9c7b-c15f8b3ca196 None admin] RBAC: Adding query filter params (interface=public, service_id=ae9d556f2df1461096a10fc17c7ffedc, region_id=RegionOne) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:27.939783 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0cd33b-b25a-4d6d-9c7b-c15f8b3ca196 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:27.941815 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0cd33b-b25a-4d6d-9c7b-c15f8b3ca196 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:27.957217 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 375/781] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Aug 29 20:20:27 2022] GET /identity/v3/endpoints?service_id=ae9d556f2df1461096a10fc17c7ffedc&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:27.957217 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0cd33b-b25a-4d6d-9c7b-c15f8b3ca196 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:27.963154 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:27.970236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20814b97-b846-4466-b942-376962f5b415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:27.970923 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20814b97-b846-4466-b942-376962f5b415 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:27.971207 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20814b97-b846-4466-b942-376962f5b415 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:27.971481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20814b97-b846-4466-b942-376962f5b415 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:27.972213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20814b97-b846-4466-b942-376962f5b415 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:27.972521 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20814b97-b846-4466-b942-376962f5b415 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:27.973957 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20814b97-b846-4466-b942-376962f5b415 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:27.988448 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 407/782] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:27.989486 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20814b97-b846-4466-b942-376962f5b415 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:28.099435 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0cd33b-b25a-4d6d-9c7b-c15f8b3ca196 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:28.119035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20814b97-b846-4466-b942-376962f5b415 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:28.138926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176c38d-5895-42ed-aeee-66b96adfdca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:28.139669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176c38d-5895-42ed-aeee-66b96adfdca3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:28.140049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176c38d-5895-42ed-aeee-66b96adfdca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:28.140396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176c38d-5895-42ed-aeee-66b96adfdca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:28.153780 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176c38d-5895-42ed-aeee-66b96adfdca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:28.155565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4176c38d-5895-42ed-aeee-66b96adfdca3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:28.161493 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6a579-ed25-48e8-bae1-65f25820cabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:28.162222 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6a579-ed25-48e8-bae1-65f25820cabb None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:28.162620 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6a579-ed25-48e8-bae1-65f25820cabb None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:28.163010 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6a579-ed25-48e8-bae1-65f25820cabb None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:28.164453 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6a579-ed25-48e8-bae1-65f25820cabb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:28.166413 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6a579-ed25-48e8-bae1-65f25820cabb None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:28.179966 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 408/783] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Aug 29 20:20:27 2022] GET /identity/v3/services => generated 772 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:28.179966 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6a579-ed25-48e8-bae1-65f25820cabb None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:28.181626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-4176c38d-5895-42ed-aeee-66b96adfdca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:28.181626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:28.181626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:28.181626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:28.181626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:28.181626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:28.181626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:28.181626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:28.181626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:28.181626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:28.181626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:28.181626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:28.181626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:28.181626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:28.181626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:28.181626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:28.182532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:28.182532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:28.182532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:28.182532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:28.182532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:28.182532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:28.182532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:28.182532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:28.182532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:28.182532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:28.182532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:28.182532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:28.182532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:28.182532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:28.182532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:28.182532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:28.182532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:28.183396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:28.183396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:28.183396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:28.186469 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 376/784] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:28.192378 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176c38d-5895-42ed-aeee-66b96adfdca3 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:28.219965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:28.275674 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:28.389352 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176c38d-5895-42ed-aeee-66b96adfdca3 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:28.405799 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6a579-ed25-48e8-bae1-65f25820cabb None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:28.446833 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537abf5e-886a-418c-9b1e-5b0e545cc5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:28.447566 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537abf5e-886a-418c-9b1e-5b0e545cc5cf None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:28.447955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537abf5e-886a-418c-9b1e-5b0e545cc5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:28.448267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537abf5e-886a-418c-9b1e-5b0e545cc5cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:28.449013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537abf5e-886a-418c-9b1e-5b0e545cc5cf None admin] RBAC: Adding query filter params (role.id=04bc48cc5b6842029b93a845d6744985, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=61709ccc1e244185b252179451cd860f) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:28.449891 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537abf5e-886a-418c-9b1e-5b0e545cc5cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:28.450426 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932b031-1680-41f7-b456-0652e5aba9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:28.451336 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537abf5e-886a-418c-9b1e-5b0e545cc5cf None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:28.451719 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932b031-1680-41f7-b456-0652e5aba9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:28.452267 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932b031-1680-41f7-b456-0652e5aba9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:28.452862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932b031-1680-41f7-b456-0652e5aba9d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:28.453913 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932b031-1680-41f7-b456-0652e5aba9d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:28.454451 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932b031-1680-41f7-b456-0652e5aba9d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:28.456665 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932b031-1680-41f7-b456-0652e5aba9d5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:28.459490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 409/785] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:20:28 2022] GET /identity/v3/role_assignments?user.id=61709ccc1e244185b252179451cd860f&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=04bc48cc5b6842029b93a845d6744985 => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:28.475179 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 377/786] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:28.481363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537abf5e-886a-418c-9b1e-5b0e545cc5cf None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:28.615780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:28.647138 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537abf5e-886a-418c-9b1e-5b0e545cc5cf None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:28.694291 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ebe25-9af7-4295-bfe6-7c76177cd4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:28.695172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ebe25-9af7-4295-bfe6-7c76177cd4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:28.695461 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ebe25-9af7-4295-bfe6-7c76177cd4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:28.695738 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ebe25-9af7-4295-bfe6-7c76177cd4b9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:28.705984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ebe25-9af7-4295-bfe6-7c76177cd4b9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:28.708223 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ebe25-9af7-4295-bfe6-7c76177cd4b9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:28.726390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-e42ebe25-9af7-4295-bfe6-7c76177cd4b9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:20:28.726390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:28.726390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:28.726390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:28.726390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:28.726390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:28.726390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:28.726390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:28.726390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:28.726390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:28.726390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:28.726390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:28.726390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:28.726390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:28.726390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:28.726390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:28.727563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:28.727563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:28.727563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:28.727563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:28.727563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:28.727563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:28.727563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:28.727563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:28.727563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:28.727563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:28.727563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:28.727563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:28.727563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:28.727563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:28.727563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:28.727563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:28.727563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:28.727563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:28.728621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:28.728621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:28.728621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:28.728621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:28.728621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:28.728621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:28.728621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:28.728621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:28.728621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:20:28.728621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:28.728621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 410/787] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:20:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:28.733335 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932b031-1680-41f7-b456-0652e5aba9d5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:28.894609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932b031-1680-41f7-b456-0652e5aba9d5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:28.910903 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:28.932462 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86738ff-4b15-4a9e-a59b-4c6aaf250670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:28.933073 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86738ff-4b15-4a9e-a59b-4c6aaf250670 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:28.933354 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86738ff-4b15-4a9e-a59b-4c6aaf250670 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:28.933719 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86738ff-4b15-4a9e-a59b-4c6aaf250670 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:28.934518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86738ff-4b15-4a9e-a59b-4c6aaf250670 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:28.934824 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86738ff-4b15-4a9e-a59b-4c6aaf250670 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:28.936297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86738ff-4b15-4a9e-a59b-4c6aaf250670 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:28.956278 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 378/788] 10.222.0.18 () {56 vars in 1135 bytes} [Mon Aug 29 20:20:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:28.961864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f775d-087a-4f0b-9de6-8ff9a50c5b14 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:28.962425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f775d-087a-4f0b-9de6-8ff9a50c5b14 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:28.962909 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f775d-087a-4f0b-9de6-8ff9a50c5b14 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:28.964719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 411/789] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:28.969224 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86738ff-4b15-4a9e-a59b-4c6aaf250670 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:28.972503 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e3419-c53c-413c-9e35-2b7f814d0217 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:28.973149 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e3419-c53c-413c-9e35-2b7f814d0217 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:28.973629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e3419-c53c-413c-9e35-2b7f814d0217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:29.025276 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-be6e3419-c53c-413c-9e35-2b7f814d0217 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:29.026596 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-be6e3419-c53c-413c-9e35-2b7f814d0217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:29.082589 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86738ff-4b15-4a9e-a59b-4c6aaf250670 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:29.154093 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02fb30-67f0-4af4-a15f-169196a821a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:29.154740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02fb30-67f0-4af4-a15f-169196a821a2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:29.155019 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02fb30-67f0-4af4-a15f-169196a821a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:29.155293 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02fb30-67f0-4af4-a15f-169196a821a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:29.162900 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 412/790] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:20:29.172213 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02fb30-67f0-4af4-a15f-169196a821a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:29.173675 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02fb30-67f0-4af4-a15f-169196a821a2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:29.188858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-8d02fb30-67f0-4af4-a15f-169196a821a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:29.188858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:29.188858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:29.188858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:29.188858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:29.188858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:29.188858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:29.188858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:29.188858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:29.188858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:29.188858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:29.188858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:29.188858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:29.188858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:29.188858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:29.188858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:29.189832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:29.189832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:29.189832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:29.189832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:29.189832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:29.189832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:29.189832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:29.189832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:29.189832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:29.189832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:29.189832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:29.189832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:29.189832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:29.189832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:29.189832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:29.189832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:29.189832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:29.190709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:29.190709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:29.190709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:29.190709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 379/791] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:29.197140 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e3419-c53c-413c-9e35-2b7f814d0217 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:29.295314 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:29.345710 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e3419-c53c-413c-9e35-2b7f814d0217 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:29.384235 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ef25c-18e0-4e2c-b598-b7e196c26096 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:29.384886 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ef25c-18e0-4e2c-b598-b7e196c26096 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:29.385361 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ef25c-18e0-4e2c-b598-b7e196c26096 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:29.387886 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 380/792] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:29.394018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62643ae4-6a2c-49ee-8b2d-eb6d1850c44a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:29.394649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62643ae4-6a2c-49ee-8b2d-eb6d1850c44a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:29.395123 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62643ae4-6a2c-49ee-8b2d-eb6d1850c44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:29.439363 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-62643ae4-6a2c-49ee-8b2d-eb6d1850c44a None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:29.447964 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-62643ae4-6a2c-49ee-8b2d-eb6d1850c44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:29.481223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:29.508920 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3371f-9296-4c01-9af2-3618d6b6a34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:29.509541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3371f-9296-4c01-9af2-3618d6b6a34e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:29.510070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3371f-9296-4c01-9af2-3618d6b6a34e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:29.510196 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3371f-9296-4c01-9af2-3618d6b6a34e None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:29.510870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3371f-9296-4c01-9af2-3618d6b6a34e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:29.511101 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3371f-9296-4c01-9af2-3618d6b6a34e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:29.512534 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3371f-9296-4c01-9af2-3618d6b6a34e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:29.525519 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 413/793] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:29.526348 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3371f-9296-4c01-9af2-3618d6b6a34e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:29.550929 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 381/794] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:20:29.561017 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62643ae4-6a2c-49ee-8b2d-eb6d1850c44a None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:29.643352 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3371f-9296-4c01-9af2-3618d6b6a34e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:29.682911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed207389-e4a0-4849-b5c1-a39aaa746572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:29.683615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed207389-e4a0-4849-b5c1-a39aaa746572 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:29.683902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed207389-e4a0-4849-b5c1-a39aaa746572 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:29.684698 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed207389-e4a0-4849-b5c1-a39aaa746572 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:29.685299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed207389-e4a0-4849-b5c1-a39aaa746572 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:29.686840 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed207389-e4a0-4849-b5c1-a39aaa746572 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:29.689045 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:29.699765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-ed207389-e4a0-4849-b5c1-a39aaa746572 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:20:29.699765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:29.699765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:29.699765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:29.699765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:29.699765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:29.699765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:29.699765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:29.699765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:20:29.699765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:29.699765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:29.699765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:29.699765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:29.699765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:29.699765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:29.699765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:29.699765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:29.700868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:29.700868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:29.700868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:29.700868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:29.700868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:29.700868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:29.700868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:29.700868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:29.700868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:29.700868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:29.700868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:29.700868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:29.700868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:29.700868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:29.700868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:29.700868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:29.701696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:29.701696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:29.701696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:29.701696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:29.701696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:29.701696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:29.701696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:29.701696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:29.701696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:29.701696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:29.701696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:29.701696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:29.701696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 29 20:20:29.701696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:29.701696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 414/795] 10.222.0.18 () {56 vars in 1129 bytes} [Mon Aug 29 20:20:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:29.706589 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed207389-e4a0-4849-b5c1-a39aaa746572 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:29.714305 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62643ae4-6a2c-49ee-8b2d-eb6d1850c44a None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:29.756619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab29ad-e325-48f3-8531-cd65cff45ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:29.758405 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab29ad-e325-48f3-8531-cd65cff45ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:29.758928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab29ad-e325-48f3-8531-cd65cff45ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:29.759428 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab29ad-e325-48f3-8531-cd65cff45ca6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:29.767920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab29ad-e325-48f3-8531-cd65cff45ca6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:29.770490 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab29ad-e325-48f3-8531-cd65cff45ca6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:29.784455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-e3ab29ad-e325-48f3-8531-cd65cff45ca6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:29.784455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:29.784455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:29.784455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:29.784455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:29.784455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:29.784455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:29.784455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:29.784455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:29.784455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:29.784455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:29.784455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:29.784455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:29.784455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:29.784455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:29.785391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:29.785391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:29.785391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:29.785391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:29.785391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:29.785391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:29.785391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:29.785391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:29.785391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:29.785391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:29.785391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:29.785391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:29.785391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:29.785391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:29.785391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:29.785391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:29.785391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:29.786287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:29.786287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:29.786287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:29.786287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:29.786287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:29.786287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:29.786287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:29.786287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:29.789528 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 382/796] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:29.795360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab29ad-e325-48f3-8531-cd65cff45ca6 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:29.828462 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:29.908880 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed207389-e4a0-4849-b5c1-a39aaa746572 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:29.945004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561fc7ab-e965-423e-a10a-bdfd92c419c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:29.945625 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fc7ab-e965-423e-a10a-bdfd92c419c5 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:29.945908 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fc7ab-e965-423e-a10a-bdfd92c419c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:29.946239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fc7ab-e965-423e-a10a-bdfd92c419c5 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:29.946881 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561fc7ab-e965-423e-a10a-bdfd92c419c5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:29.947207 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561fc7ab-e965-423e-a10a-bdfd92c419c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:29.948628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561fc7ab-e965-423e-a10a-bdfd92c419c5 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:29.953680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 415/797] 10.222.0.18 () {56 vars in 1124 bytes} [Mon Aug 29 20:20:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:29.959030 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561fc7ab-e965-423e-a10a-bdfd92c419c5 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:30.001763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab29ad-e325-48f3-8531-cd65cff45ca6 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:30.028586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:30.050368 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016a7e0-38fd-4ee6-a71d-c296bdef2cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:30.051751 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016a7e0-38fd-4ee6-a71d-c296bdef2cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:30.052317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016a7e0-38fd-4ee6-a71d-c296bdef2cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:30.052760 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016a7e0-38fd-4ee6-a71d-c296bdef2cf7 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:30.053852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016a7e0-38fd-4ee6-a71d-c296bdef2cf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:30.054778 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016a7e0-38fd-4ee6-a71d-c296bdef2cf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:30.056714 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016a7e0-38fd-4ee6-a71d-c296bdef2cf7 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:30.073801 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 383/798] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:30.074898 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016a7e0-38fd-4ee6-a71d-c296bdef2cf7 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:30.077461 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561fc7ab-e965-423e-a10a-bdfd92c419c5 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:30.120054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9207d-26c6-460a-a048-e07b490d2065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:30.120862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9207d-26c6-460a-a048-e07b490d2065 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:30.121301 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9207d-26c6-460a-a048-e07b490d2065 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:30.121641 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9207d-26c6-460a-a048-e07b490d2065 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:30.122353 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9207d-26c6-460a-a048-e07b490d2065 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:30.122724 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9207d-26c6-460a-a048-e07b490d2065 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:30.124272 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9207d-26c6-460a-a048-e07b490d2065 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:30.130868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 416/799] 10.222.0.18 () {56 vars in 1124 bytes} [Mon Aug 29 20:20:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:30.131889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9207d-26c6-460a-a048-e07b490d2065 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:30.181061 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:30.213119 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016a7e0-38fd-4ee6-a71d-c296bdef2cf7 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:30.250247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b460ab6-4b1b-46f1-97d7-914c1a5a489b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:30.251018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b460ab6-4b1b-46f1-97d7-914c1a5a489b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:30.251326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b460ab6-4b1b-46f1-97d7-914c1a5a489b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:30.251717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b460ab6-4b1b-46f1-97d7-914c1a5a489b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:30.261054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9207d-26c6-460a-a048-e07b490d2065 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:30.264114 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b460ab6-4b1b-46f1-97d7-914c1a5a489b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:30.269125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-5b460ab6-4b1b-46f1-97d7-914c1a5a489b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:20:30.269125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:30.269125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:30.269125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:30.269125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:30.269125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:30.269125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:30.269125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:30.269125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:30.269125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:30.269125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:30.269125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:30.269125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:30.269125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:30.269125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:30.269125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:30.270370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:30.270370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:30.270370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:30.270370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:30.270370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:30.270370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:30.270370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:30.270370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:30.270370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:30.270370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:30.270370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:30.270370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:30.270370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:30.270370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:30.270370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:30.270370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:30.271379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:30.271379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:30.271379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:30.271379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:30.271379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:30.271379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:30.271379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:30.271379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:30.271379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:30.271379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:30.271379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:20:30.271379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:30.276214 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 384/800] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:20:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:30.277099 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b460ab6-4b1b-46f1-97d7-914c1a5a489b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:30.314540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:30.332166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a2d96-9be1-4b5d-b9f5-2c1a8728c2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:30.333187 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a2d96-9be1-4b5d-b9f5-2c1a8728c2a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:30.333934 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a2d96-9be1-4b5d-b9f5-2c1a8728c2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:30.334360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a2d96-9be1-4b5d-b9f5-2c1a8728c2a1 None admin] PATH_INFO: `/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/599129b7d8ff47efb145787414ddb80d/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:30.337657 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a2d96-9be1-4b5d-b9f5-2c1a8728c2a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=90b25dc95d5a4a068994bdd801d6b8ea, user_id=599129b7d8ff47efb145787414ddb80d, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:30.339340 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a2d96-9be1-4b5d-b9f5-2c1a8728c2a1 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:30.357677 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 417/801] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:20:30 2022] PUT /identity/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/599129b7d8ff47efb145787414ddb80d/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:20:30.358827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a2d96-9be1-4b5d-b9f5-2c1a8728c2a1 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:30.475070 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b460ab6-4b1b-46f1-97d7-914c1a5a489b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:30.499256 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a2d96-9be1-4b5d-b9f5-2c1a8728c2a1 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:30.545138 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa7401-4535-4455-ad04-93fdd26b01ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:30.546047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa7401-4535-4455-ad04-93fdd26b01ec None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:30.546475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa7401-4535-4455-ad04-93fdd26b01ec None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:30.547162 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa7401-4535-4455-ad04-93fdd26b01ec None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:30.548512 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa7401-4535-4455-ad04-93fdd26b01ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:30.549021 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa7401-4535-4455-ad04-93fdd26b01ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:30.550865 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa7401-4535-4455-ad04-93fdd26b01ec None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:30.561836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd0594-37ba-470d-8614-2f88870e9b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:30.562554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd0594-37ba-470d-8614-2f88870e9b0b None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:30.562904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd0594-37ba-470d-8614-2f88870e9b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:30.563174 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd0594-37ba-470d-8614-2f88870e9b0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:30.563573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 418/802] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:20:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:30.563848 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd0594-37ba-470d-8614-2f88870e9b0b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:30.565231 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd0594-37ba-470d-8614-2f88870e9b0b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:30.574533 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa7401-4535-4455-ad04-93fdd26b01ec None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:30.575358 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:30.609476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 385/803] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:20:30 2022] POST /identity/v3/endpoints => generated 346 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:20:30.669765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:30.753019 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa7401-4535-4455-ad04-93fdd26b01ec None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:30.799893 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832b46b-0dc1-4abf-891e-7b8d8b3ea38c None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:30.800993 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832b46b-0dc1-4abf-891e-7b8d8b3ea38c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:30.801468 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832b46b-0dc1-4abf-891e-7b8d8b3ea38c None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:30.812460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 386/804] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:30 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:30.818884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaada20-0618-441e-a692-f622a52374e9 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:30.819186 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaada20-0618-441e-a692-f622a52374e9 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:30.819441 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaada20-0618-441e-a692-f622a52374e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:30.835417 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fbe8a-adad-4b46-a28a-d7662abe7cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:30.836187 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fbe8a-adad-4b46-a28a-d7662abe7cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:30.836699 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fbe8a-adad-4b46-a28a-d7662abe7cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:30.837049 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fbe8a-adad-4b46-a28a-d7662abe7cf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:30.843350 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-2aaada20-0618-441e-a692-f622a52374e9 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:30.844885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2aaada20-0618-441e-a692-f622a52374e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:30.850195 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18fbe8a-adad-4b46-a28a-d7662abe7cf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:30.851861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18fbe8a-adad-4b46-a28a-d7662abe7cf7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:30.870506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-d18fbe8a-adad-4b46-a28a-d7662abe7cf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:30.870506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:30.870506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:30.870506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:30.870506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:30.870506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:30.870506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:30.870506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:30.870506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:30.870506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:30.870506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:30.870506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:30.870506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:30.870506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:30.870506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:30.870506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:30.871743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:30.871743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:30.871743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:30.871743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:30.871743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:30.871743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:30.871743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:30.871743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:30.871743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:30.871743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:30.871743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:30.871743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:30.871743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:30.871743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:30.871743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:30.871743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:30.871743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:30.872829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:30.872829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:30.872829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:30.872829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 419/805] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:30.873760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c60a0-e555-450d-9b36-bc15a1dea70b None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:30.874046 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c60a0-e555-450d-9b36-bc15a1dea70b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:30.874319 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c60a0-e555-450d-9b36-bc15a1dea70b None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:30.875804 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 420/806] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:30.880680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c60a0-e555-450d-9b36-bc15a1dea70b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:30.942892 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 387/807] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:20:30.944779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a666f-34b5-4d49-abb2-342f9a283c1f None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:30.945314 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a666f-34b5-4d49-abb2-342f9a283c1f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:30.945700 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a666f-34b5-4d49-abb2-342f9a283c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:30.975988 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-df0a666f-34b5-4d49-abb2-342f9a283c1f None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:30.978904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-df0a666f-34b5-4d49-abb2-342f9a283c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:31.044449 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:31.055559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c60a0-e555-450d-9b36-bc15a1dea70b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:31.097978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 388/808] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:31.117831 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da148149-7185-45ad-8f14-2dfa369b4d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:31.118455 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da148149-7185-45ad-8f14-2dfa369b4d4d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:31.118737 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da148149-7185-45ad-8f14-2dfa369b4d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:31.119017 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da148149-7185-45ad-8f14-2dfa369b4d4d None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:31.119672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da148149-7185-45ad-8f14-2dfa369b4d4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:31.119996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da148149-7185-45ad-8f14-2dfa369b4d4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:31.121410 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da148149-7185-45ad-8f14-2dfa369b4d4d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:31.134366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 421/809] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:31.141085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a666f-34b5-4d49-abb2-342f9a283c1f None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:31.204687 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:31.273970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a666f-34b5-4d49-abb2-342f9a283c1f None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:31.326920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c28d9-dc91-4068-8bb9-6fbadc97ad54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:31.328697 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c28d9-dc91-4068-8bb9-6fbadc97ad54 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:31.328974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c28d9-dc91-4068-8bb9-6fbadc97ad54 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:31.329241 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c28d9-dc91-4068-8bb9-6fbadc97ad54 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:31.340761 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022c28d9-dc91-4068-8bb9-6fbadc97ad54 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:31.342228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022c28d9-dc91-4068-8bb9-6fbadc97ad54 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:31.357710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-022c28d9-dc91-4068-8bb9-6fbadc97ad54 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:20:31.357710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:31.357710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:31.357710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:31.357710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:31.357710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:31.357710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:31.357710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:31.357710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:31.357710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:31.357710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:31.357710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:31.357710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:31.357710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:31.357710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:31.357710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:31.359196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:31.359196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:31.359196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:31.359196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:31.359196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:31.359196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:31.359196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:31.359196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:31.359196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:31.359196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:31.359196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:31.359196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:31.359196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:31.359196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:31.359196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:31.359196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:31.359196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:31.359196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:31.360565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:31.360565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:31.360565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:31.360565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:31.360565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:31.360565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:31.360565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:31.360565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:31.360565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:20:31.360565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:31.360565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 389/810] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Aug 29 20:20:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:31.366922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da148149-7185-45ad-8f14-2dfa369b4d4d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:31.415632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c28d9-dc91-4068-8bb9-6fbadc97ad54 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:31.503844 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da148149-7185-45ad-8f14-2dfa369b4d4d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:31.541882 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88088bb-0866-4dab-9ae7-8c5744c6e6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:31.542601 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88088bb-0866-4dab-9ae7-8c5744c6e6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:31.542947 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88088bb-0866-4dab-9ae7-8c5744c6e6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:31.543222 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88088bb-0866-4dab-9ae7-8c5744c6e6f9 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:31.544278 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88088bb-0866-4dab-9ae7-8c5744c6e6f9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:31.544465 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88088bb-0866-4dab-9ae7-8c5744c6e6f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:31.545888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88088bb-0866-4dab-9ae7-8c5744c6e6f9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:31.567272 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c28d9-dc91-4068-8bb9-6fbadc97ad54 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:31.567776 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 422/811] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Aug 29 20:20:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:31.568837 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88088bb-0866-4dab-9ae7-8c5744c6e6f9 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:31.579241 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:31.591612 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:31.606404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b71e8-e109-4696-b16d-8af5b47e8e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:31.607139 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b71e8-e109-4696-b16d-8af5b47e8e90 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:31.607489 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b71e8-e109-4696-b16d-8af5b47e8e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:31.607834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b71e8-e109-4696-b16d-8af5b47e8e90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:31.622752 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b71e8-e109-4696-b16d-8af5b47e8e90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:31.629876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-975b71e8-e109-4696-b16d-8af5b47e8e90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:31.629876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:31.629876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:31.629876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:31.629876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:31.629876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:31.629876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:31.629876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:31.629876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:31.629876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:31.629876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:31.629876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:31.629876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:31.629876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:31.629876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:31.629876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:31.631897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:31.631897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:31.631897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:31.631897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:31.631897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:31.631897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:31.631897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:31.631897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:31.631897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:31.631897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:31.631897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:31.631897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:31.631897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:31.631897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:31.631897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:31.631897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:31.633444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:31.633444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:31.633444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:31.633444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:31.633444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:31.633444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:31.633444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:31.633444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:31.633444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:31.633444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:31.633444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:31.633444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:31.633444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 390/812] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:20:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:31.633444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b71e8-e109-4696-b16d-8af5b47e8e90 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:31.701004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88088bb-0866-4dab-9ae7-8c5744c6e6f9 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:31.738406 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7160de-42f5-43ac-9a61-da7f44a313bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:31.739187 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7160de-42f5-43ac-9a61-da7f44a313bc None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:31.739530 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7160de-42f5-43ac-9a61-da7f44a313bc None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:31.739915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7160de-42f5-43ac-9a61-da7f44a313bc None admin] PATH_INFO: `/v3/services/network` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:31.741102 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7160de-42f5-43ac-9a61-da7f44a313bc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:31.742548 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7160de-42f5-43ac-9a61-da7f44a313bc None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:31.755095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-0c7160de-42f5-43ac-9a61-da7f44a313bc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:20:31.755095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:31.755095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:31.755095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:31.755095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:31.755095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:31.755095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:31.755095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:31.755095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:20:31.755095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:31.755095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:31.755095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:31.755095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:31.755095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:31.755095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:31.755095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:31.755095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:31.757263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:31.757263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:31.757263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:31.757263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:31.757263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:31.757263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:31.757263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:31.757263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:31.757263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:31.757263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:31.757263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:31.757263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:31.757263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:31.757263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:31.757263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:31.757263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:31.759211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:31.759211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:31.759211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:31.759211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:31.759211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:31.759211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:31.759211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:31.759211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:31.759211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:31.759211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:31.759211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:31.759211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:31.759211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 29 20:20:31.759211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:31.760922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 423/813] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:31 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:31.760922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7160de-42f5-43ac-9a61-da7f44a313bc None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:31.774147 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b71e8-e109-4696-b16d-8af5b47e8e90 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:31.822313 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ee9ca-00b0-4114-8656-0d7f4ddf4835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:31.823332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ee9ca-00b0-4114-8656-0d7f4ddf4835 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:31.823657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ee9ca-00b0-4114-8656-0d7f4ddf4835 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:31.825352 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ee9ca-00b0-4114-8656-0d7f4ddf4835 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:31.863722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:31.883502 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:31.893109 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7160de-42f5-43ac-9a61-da7f44a313bc None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:31.929808 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ee9ca-00b0-4114-8656-0d7f4ddf4835 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:31.931640 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ee9ca-00b0-4114-8656-0d7f4ddf4835 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:31.934293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b54d0-eb69-442a-8f2a-a825d71ac594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:31.935020 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b54d0-eb69-442a-8f2a-a825d71ac594 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:31.935423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b54d0-eb69-442a-8f2a-a825d71ac594 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:31.935672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b54d0-eb69-442a-8f2a-a825d71ac594 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:31.936390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b54d0-eb69-442a-8f2a-a825d71ac594 None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:31.936699 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b54d0-eb69-442a-8f2a-a825d71ac594 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:31.938190 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b54d0-eb69-442a-8f2a-a825d71ac594 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:31.951899 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 424/814] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:20:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:31.951899 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b54d0-eb69-442a-8f2a-a825d71ac594 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:32.082906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 391/815] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:20:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:20:32.084180 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ee9ca-00b0-4114-8656-0d7f4ddf4835 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:32.085497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b54d0-eb69-442a-8f2a-a825d71ac594 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:32.132502 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6f9bf-b608-4d72-ae6a-6cac07aa21d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:32.133171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6f9bf-b608-4d72-ae6a-6cac07aa21d8 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:32.133446 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6f9bf-b608-4d72-ae6a-6cac07aa21d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:32.133728 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6f9bf-b608-4d72-ae6a-6cac07aa21d8 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:32.134377 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6f9bf-b608-4d72-ae6a-6cac07aa21d8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:32.134686 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6f9bf-b608-4d72-ae6a-6cac07aa21d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:32.136103 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6f9bf-b608-4d72-ae6a-6cac07aa21d8 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:32.142537 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 425/816] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 29 20:20:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:32.147829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6f9bf-b608-4d72-ae6a-6cac07aa21d8 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:32.150031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:32.188519 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:32.241951 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ee9ca-00b0-4114-8656-0d7f4ddf4835 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:32.316945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b31273-c2a0-46ef-9fed-b0c8537dbba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:32.318753 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b31273-c2a0-46ef-9fed-b0c8537dbba8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:32.319770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b31273-c2a0-46ef-9fed-b0c8537dbba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:32.320479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b31273-c2a0-46ef-9fed-b0c8537dbba8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:32.323038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6f9bf-b608-4d72-ae6a-6cac07aa21d8 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:32.338892 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b31273-c2a0-46ef-9fed-b0c8537dbba8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:32.342414 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b31273-c2a0-46ef-9fed-b0c8537dbba8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:32.368069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-58b31273-c2a0-46ef-9fed-b0c8537dbba8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:32.368069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:32.368069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:32.368069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:32.368069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:32.368069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:32.368069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:32.368069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:32.368069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:32.368069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:32.368069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:32.368069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:32.368069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:32.368069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:32.368069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:32.368069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:32.369049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:32.369049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:32.369049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:32.369049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:32.369049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:32.369049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:32.369049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:32.369049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:32.369049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:32.369049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:32.369049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:32.369049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:32.369049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:32.369049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:32.369049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:32.369049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:32.369049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:32.369913 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:32.369913 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:32.369913 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:32.371462 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c659545-fe73-4e06-a989-91f89dcf4340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:32.372206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c659545-fe73-4e06-a989-91f89dcf4340 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:32.372585 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c659545-fe73-4e06-a989-91f89dcf4340 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:32.373662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c659545-fe73-4e06-a989-91f89dcf4340 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:32.377853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 392/817] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:32.377853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b31273-c2a0-46ef-9fed-b0c8537dbba8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:32.393613 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c659545-fe73-4e06-a989-91f89dcf4340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:32.396995 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c659545-fe73-4e06-a989-91f89dcf4340 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:32.425756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-8c659545-fe73-4e06-a989-91f89dcf4340 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:32.425756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:32.425756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:32.425756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:32.425756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:32.425756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:32.425756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:32.425756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:32.425756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:32.425756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:32.425756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:32.425756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:32.425756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:32.425756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:32.425756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:32.425756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:32.428525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:32.428525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:32.428525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:32.428525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:32.428525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:32.428525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:32.428525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:32.428525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:32.428525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:32.428525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:32.428525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:32.428525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:32.428525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:32.428525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:32.428525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:32.428525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:32.428525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:32.429421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:32.429421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:32.429421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:32.434413 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 426/818] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:32.436529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c659545-fe73-4e06-a989-91f89dcf4340 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:32.558011 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:32.614640 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c659545-fe73-4e06-a989-91f89dcf4340 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:32.641647 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b31273-c2a0-46ef-9fed-b0c8537dbba8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:32.663622 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3d892-f9c4-419f-920b-73fd2ffc2819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:32.664926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3d892-f9c4-419f-920b-73fd2ffc2819 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:32.665428 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3d892-f9c4-419f-920b-73fd2ffc2819 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:32.665953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3d892-f9c4-419f-920b-73fd2ffc2819 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:32.666893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f3d892-f9c4-419f-920b-73fd2ffc2819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:32.667430 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f3d892-f9c4-419f-920b-73fd2ffc2819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:32.668643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:32.670499 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f3d892-f9c4-419f-920b-73fd2ffc2819 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:32.682498 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc45d8-04b0-401d-8f39-bbe72f6a30c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:32.683502 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc45d8-04b0-401d-8f39-bbe72f6a30c5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:32.683502 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc45d8-04b0-401d-8f39-bbe72f6a30c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:32.683694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc45d8-04b0-401d-8f39-bbe72f6a30c5 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:32.684338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc45d8-04b0-401d-8f39-bbe72f6a30c5 None admin] RBAC: Adding query filter params (type=network) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:32.684700 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc45d8-04b0-401d-8f39-bbe72f6a30c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:32.686408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc45d8-04b0-401d-8f39-bbe72f6a30c5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:32.696741 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 393/820] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 29 20:20:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:32.697106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 427/820] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:32.697879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc45d8-04b0-401d-8f39-bbe72f6a30c5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:32.700647 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3d892-f9c4-419f-920b-73fd2ffc2819 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:32.832949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc45d8-04b0-401d-8f39-bbe72f6a30c5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:32.835700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3d892-f9c4-419f-920b-73fd2ffc2819 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:32.861042 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:32.874009 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bc862-ef1d-446d-975b-ba034f03a6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:32.874534 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bc862-ef1d-446d-975b-ba034f03a6a8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:32.874822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bc862-ef1d-446d-975b-ba034f03a6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:32.875092 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bc862-ef1d-446d-975b-ba034f03a6a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:32.875734 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3bc862-ef1d-446d-975b-ba034f03a6a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:32.876076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3bc862-ef1d-446d-975b-ba034f03a6a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:32.877453 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3bc862-ef1d-446d-975b-ba034f03a6a8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:32.884967 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e800fd8-0fe0-4298-95c6-142c5945ab21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:32.885720 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e800fd8-0fe0-4298-95c6-142c5945ab21 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:32.886003 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e800fd8-0fe0-4298-95c6-142c5945ab21 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:32.886283 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e800fd8-0fe0-4298-95c6-142c5945ab21 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:32.892287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 394/821] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:32.893118 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bc862-ef1d-446d-975b-ba034f03a6a8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:32.901432 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e800fd8-0fe0-4298-95c6-142c5945ab21 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:32.903523 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e800fd8-0fe0-4298-95c6-142c5945ab21 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:32.917136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-6e800fd8-0fe0-4298-95c6-142c5945ab21 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:20:32.917136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:32.917136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:32.917136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:32.917136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:32.917136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:32.917136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:32.917136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:32.917136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:32.917136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:32.917136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:32.917136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:32.917136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:32.917136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:32.917136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:32.917136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:32.918133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:32.918133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:32.918133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:32.918133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:32.918133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:32.918133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:32.918133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:32.918133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:32.918133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:32.918133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:32.918133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:32.918133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:32.918133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:32.918133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:32.918133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:32.918133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:32.918133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:32.918133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:32.919156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:32.919156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:32.919156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:32.919156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:32.919156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:32.919156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:32.919156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:32.919156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:32.919156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:20:32.919156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:32.920146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 428/822] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Aug 29 20:20:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:32.921554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d94cfe-3c0a-4e5b-8cce-24f835cf777a None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:32.921843 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d94cfe-3c0a-4e5b-8cce-24f835cf777a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:32.922121 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d94cfe-3c0a-4e5b-8cce-24f835cf777a None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:32.923394 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 429/823] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:32.924442 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36da6c7-30e5-4af2-b763-110ba06d65d5 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:32.924723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36da6c7-30e5-4af2-b763-110ba06d65d5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:32.924994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36da6c7-30e5-4af2-b763-110ba06d65d5 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:32.926569 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 430/824] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:32.928698 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36da6c7-30e5-4af2-b763-110ba06d65d5 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:32.943364 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:33.000824 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bc862-ef1d-446d-975b-ba034f03a6a8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:33.039927 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36da6c7-30e5-4af2-b763-110ba06d65d5 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:33.042913 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc850b7-7253-495f-a5ff-b5a157d53d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:33.043534 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc850b7-7253-495f-a5ff-b5a157d53d35 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:33.043819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc850b7-7253-495f-a5ff-b5a157d53d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:33.045216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc850b7-7253-495f-a5ff-b5a157d53d35 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:33.045216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc850b7-7253-495f-a5ff-b5a157d53d35 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:33.046191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc850b7-7253-495f-a5ff-b5a157d53d35 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:33.084064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 395/825] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:20:32 2022] POST /identity/v3/endpoints => generated 337 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:20:33.084064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc850b7-7253-495f-a5ff-b5a157d53d35 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:33.090559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89c5c64-b6a8-4e23-b1f6-1c1535358024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:33.091395 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c5c64-b6a8-4e23-b1f6-1c1535358024 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:33.091680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c5c64-b6a8-4e23-b1f6-1c1535358024 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:33.091980 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89c5c64-b6a8-4e23-b1f6-1c1535358024 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:33.104374 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c5c64-b6a8-4e23-b1f6-1c1535358024 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:33.106833 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89c5c64-b6a8-4e23-b1f6-1c1535358024 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:33.122629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-f89c5c64-b6a8-4e23-b1f6-1c1535358024 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:33.122629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:33.122629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:33.122629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:33.122629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:33.122629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:33.122629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:33.122629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:33.122629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:33.122629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:33.122629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:33.122629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:33.122629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:33.122629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:33.122629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:33.123666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:33.123666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:33.123666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:33.123666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:33.123666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:33.123666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:33.123666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:33.123666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:33.123666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:33.123666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:33.123666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:33.123666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:33.123666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:33.123666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:33.123666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:33.123666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:33.123666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:33.125146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:33.125146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:33.125146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:33.125146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:33.125146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:33.125146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:33.125146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:33.125146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:33.125146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 431/826] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:33.125643 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f559ee-ce63-4fec-9308-e3e2a0de57a9 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:33.126868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f559ee-ce63-4fec-9308-e3e2a0de57a9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:33.126868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f559ee-ce63-4fec-9308-e3e2a0de57a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:33.148165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a2f559ee-ce63-4fec-9308-e3e2a0de57a9 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:33.149429 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a2f559ee-ce63-4fec-9308-e3e2a0de57a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:33.187285 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:33.220825 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc850b7-7253-495f-a5ff-b5a157d53d35 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:33.222600 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:33.245899 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 432/827] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:33.247809 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee9133-a29a-457a-8c0f-09429c54ca72 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:33.248135 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee9133-a29a-457a-8c0f-09429c54ca72 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:33.248716 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee9133-a29a-457a-8c0f-09429c54ca72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:33.272168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-0fee9133-a29a-457a-8c0f-09429c54ca72 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:33.273593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0fee9133-a29a-457a-8c0f-09429c54ca72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:33.287264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89961042-f512-42ed-b0b7-192c5ee95086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:33.287264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89961042-f512-42ed-b0b7-192c5ee95086 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:33.287264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89961042-f512-42ed-b0b7-192c5ee95086 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:33.287264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89961042-f512-42ed-b0b7-192c5ee95086 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:33.287679 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89961042-f512-42ed-b0b7-192c5ee95086 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:33.287959 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89961042-f512-42ed-b0b7-192c5ee95086 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:33.290000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89961042-f512-42ed-b0b7-192c5ee95086 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:33.320310 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 396/828] 10.222.0.18 () {56 vars in 1149 bytes} [Mon Aug 29 20:20:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:33.321450 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89961042-f512-42ed-b0b7-192c5ee95086 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:33.368942 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 433/829] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:33.370011 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee9133-a29a-457a-8c0f-09429c54ca72 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:33.443166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89961042-f512-42ed-b0b7-192c5ee95086 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:33.479635 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:33.489667 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b53610-cddc-4577-904e-f0d48633963d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:33.490011 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee9133-a29a-457a-8c0f-09429c54ca72 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:33.490813 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b53610-cddc-4577-904e-f0d48633963d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:33.491301 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b53610-cddc-4577-904e-f0d48633963d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:33.491760 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b53610-cddc-4577-904e-f0d48633963d None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:33.492695 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b53610-cddc-4577-904e-f0d48633963d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:33.493213 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b53610-cddc-4577-904e-f0d48633963d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:33.495138 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b53610-cddc-4577-904e-f0d48633963d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:33.510975 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 397/830] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:33.520418 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b53610-cddc-4577-904e-f0d48633963d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:33.528606 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ff25a-6f20-46b9-b950-8006b777f02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:33.529258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ff25a-6f20-46b9-b950-8006b777f02b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:33.529544 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ff25a-6f20-46b9-b950-8006b777f02b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:33.529815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ff25a-6f20-46b9-b950-8006b777f02b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:33.541609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ff25a-6f20-46b9-b950-8006b777f02b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:33.543865 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ff25a-6f20-46b9-b950-8006b777f02b None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:33.547501 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:33.562031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-bc9ff25a-6f20-46b9-b950-8006b777f02b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:33.562031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:33.562031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:33.562031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:33.562031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:33.562031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:33.562031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:33.562031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:33.562031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:33.562031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:33.562031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:33.562031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:33.562031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:33.562031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:33.562031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:33.562031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:33.563343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:33.563343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:33.563343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:33.563343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:33.563343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:33.563343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:33.563343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:33.563343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:33.563343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:33.563343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:33.563343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:33.563343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:33.563343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:33.563343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:33.563343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:33.563343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:33.563343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:33.564493 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:33.564493 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:33.564493 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:33.564493 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 434/831] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:33.565627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ff25a-6f20-46b9-b950-8006b777f02b None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:33.663461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b53610-cddc-4577-904e-f0d48633963d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:33.695209 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ff25a-6f20-46b9-b950-8006b777f02b None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:33.702221 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db0b81-9415-4703-b28a-eb179d2fbd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:33.702838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db0b81-9415-4703-b28a-eb179d2fbd59 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:33.703117 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db0b81-9415-4703-b28a-eb179d2fbd59 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:33.703395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db0b81-9415-4703-b28a-eb179d2fbd59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:33.704161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db0b81-9415-4703-b28a-eb179d2fbd59 None admin] RBAC: Adding query filter params (role.id=322da7bd64024770a64bdb73e5c54719, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=c0c44754a1c04a2b87081aba6cb509d3) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:33.704474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db0b81-9415-4703-b28a-eb179d2fbd59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:33.705908 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db0b81-9415-4703-b28a-eb179d2fbd59 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:33.712391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 398/832] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:20:33 2022] GET /identity/v3/role_assignments?user.id=c0c44754a1c04a2b87081aba6cb509d3&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=322da7bd64024770a64bdb73e5c54719 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:33.713131 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db0b81-9415-4703-b28a-eb179d2fbd59 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:33.750626 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e34322-b66c-4eea-b053-9328190bd8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:33.751393 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e34322-b66c-4eea-b053-9328190bd8ce None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:33.751747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e34322-b66c-4eea-b053-9328190bd8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:33.752053 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e34322-b66c-4eea-b053-9328190bd8ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:33.766564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e34322-b66c-4eea-b053-9328190bd8ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:33.773315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-c7e34322-b66c-4eea-b053-9328190bd8ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:20:33.773315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:33.773315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:33.773315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:33.773315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:33.773315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:33.773315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:33.773315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:33.773315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:33.773315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:33.773315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:33.773315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:33.773315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:33.773315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:33.773315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:33.773315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:33.774507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:33.774507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:33.774507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:33.774507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:33.774507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:33.774507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:33.774507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:33.774507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:33.774507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:33.774507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:33.774507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:33.774507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:33.774507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:33.774507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:33.774507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:33.774507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:33.779054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:33.779054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:33.779054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:33.779054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:33.779054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:33.779054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:33.779054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:33.779054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:33.779054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:33.779054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:33.779054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:20:33.779054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:33.779054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 435/833] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:20:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:33.779054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e34322-b66c-4eea-b053-9328190bd8ce None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:33.781523 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:33.843435 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:33.878072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db0b81-9415-4703-b28a-eb179d2fbd59 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:33.915396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e34322-b66c-4eea-b053-9328190bd8ce None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:33.920414 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152943fc-4aae-467d-92ca-9d5303ad46c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:33.920920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152943fc-4aae-467d-92ca-9d5303ad46c6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:33.921875 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152943fc-4aae-467d-92ca-9d5303ad46c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:33.921875 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152943fc-4aae-467d-92ca-9d5303ad46c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:33.922308 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152943fc-4aae-467d-92ca-9d5303ad46c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:33.923031 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152943fc-4aae-467d-92ca-9d5303ad46c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:33.925182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152943fc-4aae-467d-92ca-9d5303ad46c6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:33.942860 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 399/834] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:33.942860 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152943fc-4aae-467d-92ca-9d5303ad46c6 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:33.962651 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60c48e-e9ec-4768-8a67-cbff9f05a78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:33.963555 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60c48e-e9ec-4768-8a67-cbff9f05a78d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:33.964064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60c48e-e9ec-4768-8a67-cbff9f05a78d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:33.964480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60c48e-e9ec-4768-8a67-cbff9f05a78d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:33.965813 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60c48e-e9ec-4768-8a67-cbff9f05a78d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:33.967348 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60c48e-e9ec-4768-8a67-cbff9f05a78d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:33.979091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-af60c48e-e9ec-4768-8a67-cbff9f05a78d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:20:33.979091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:33.979091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:33.979091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:33.979091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:33.979091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:33.979091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:33.979091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:33.979091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:20:33.979091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:33.979091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:33.979091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:33.979091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:33.979091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:33.979091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:33.979091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:33.979091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:33.980419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:33.980419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:33.980419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:33.980419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:33.980419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:33.980419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:33.980419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:33.980419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:33.980419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:33.980419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:33.980419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:33.980419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:33.980419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:33.980419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:33.980419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:33.980419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:33.981245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:33.981245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:33.981245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:33.981245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:33.981245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:33.981245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:33.981245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:33.981245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:33.981245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:33.981245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:33.981245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:33.981245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:33.981245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:20:33.981245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:33.988328 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 436/835] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:20:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:33.989121 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60c48e-e9ec-4768-8a67-cbff9f05a78d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:34.061051 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152943fc-4aae-467d-92ca-9d5303ad46c6 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:34.096097 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:34.107018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55aae3d-8546-488e-937c-dd9227d453d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:34.107698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55aae3d-8546-488e-937c-dd9227d453d8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:34.108014 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55aae3d-8546-488e-937c-dd9227d453d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:34.108310 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55aae3d-8546-488e-937c-dd9227d453d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:34.108987 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55aae3d-8546-488e-937c-dd9227d453d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:34.109291 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55aae3d-8546-488e-937c-dd9227d453d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:34.110674 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55aae3d-8546-488e-937c-dd9227d453d8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:34.120334 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 400/836] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:20:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:34.121170 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55aae3d-8546-488e-937c-dd9227d453d8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:34.131613 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:34.148809 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60c48e-e9ec-4768-8a67-cbff9f05a78d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:34.188597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91156454-9cbd-4478-9e00-09fbf5e57e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:34.189242 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91156454-9cbd-4478-9e00-09fbf5e57e47 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:34.189524 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91156454-9cbd-4478-9e00-09fbf5e57e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:34.189798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91156454-9cbd-4478-9e00-09fbf5e57e47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:34.197099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91156454-9cbd-4478-9e00-09fbf5e57e47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:34.198543 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91156454-9cbd-4478-9e00-09fbf5e57e47 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:34.211056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-91156454-9cbd-4478-9e00-09fbf5e57e47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:34.211056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:34.211056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:34.211056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:34.211056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:34.211056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:34.211056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:34.211056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:34.211056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:34.211056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:34.211056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:34.211056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:34.211056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:34.211056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:34.211056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:34.212028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:34.212028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:34.212028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:34.212028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:34.212028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:34.212028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:34.212028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:34.212028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:34.212028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:34.212028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:34.212028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:34.212028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:34.212028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:34.212028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:34.212028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:34.212028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:34.212028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:34.213483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:34.213483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:34.213483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:34.213483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:34.213483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:34.213483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:34.213483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:34.213483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:34.213483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 437/837] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:34.213483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91156454-9cbd-4478-9e00-09fbf5e57e47 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:34.253106 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55aae3d-8546-488e-937c-dd9227d453d8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:34.309486 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c942f4-f72f-42ba-a884-2dea82e54b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:34.310206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c942f4-f72f-42ba-a884-2dea82e54b00 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:34.310487 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c942f4-f72f-42ba-a884-2dea82e54b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:34.311084 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c942f4-f72f-42ba-a884-2dea82e54b00 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:34.311748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c942f4-f72f-42ba-a884-2dea82e54b00 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:34.312384 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c942f4-f72f-42ba-a884-2dea82e54b00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:34.313695 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c942f4-f72f-42ba-a884-2dea82e54b00 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:34.323894 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 401/838] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:20:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:34.324922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c942f4-f72f-42ba-a884-2dea82e54b00 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:34.348262 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91156454-9cbd-4478-9e00-09fbf5e57e47 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:34.379133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:34.390853 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7693a-1a88-4075-8d04-ecad3fea2e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:34.391540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7693a-1a88-4075-8d04-ecad3fea2e63 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:34.391821 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7693a-1a88-4075-8d04-ecad3fea2e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:34.392178 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7693a-1a88-4075-8d04-ecad3fea2e63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:34.405357 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf7693a-1a88-4075-8d04-ecad3fea2e63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:34.406944 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf7693a-1a88-4075-8d04-ecad3fea2e63 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:34.408916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:34.425593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-daf7693a-1a88-4075-8d04-ecad3fea2e63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:34.425593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:34.425593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:34.425593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:34.425593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:34.425593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:34.425593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:34.425593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:34.425593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:34.425593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:34.425593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:34.425593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:34.425593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:34.425593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:34.425593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:34.425593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:34.426584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:34.426584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:34.426584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:34.426584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:34.426584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:34.426584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:34.426584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:34.426584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:34.426584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:34.426584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:34.426584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:34.426584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:34.426584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:34.426584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:34.426584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:34.426584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:34.426584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:34.427453 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:34.427453 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:34.427453 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:34.427453 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 438/839] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:34.427680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7693a-1a88-4075-8d04-ecad3fea2e63 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:34.450593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c942f4-f72f-42ba-a884-2dea82e54b00 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:34.488201 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44161ee-a9f0-4b27-a477-ded0e5a3b6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:34.488720 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44161ee-a9f0-4b27-a477-ded0e5a3b6fe None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:34.488995 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44161ee-a9f0-4b27-a477-ded0e5a3b6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:34.489268 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44161ee-a9f0-4b27-a477-ded0e5a3b6fe None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:34.489943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44161ee-a9f0-4b27-a477-ded0e5a3b6fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:34.490243 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44161ee-a9f0-4b27-a477-ded0e5a3b6fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:34.491664 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44161ee-a9f0-4b27-a477-ded0e5a3b6fe None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:34.505464 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 402/840] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:34.506616 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44161ee-a9f0-4b27-a477-ded0e5a3b6fe None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:34.558076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7693a-1a88-4075-8d04-ecad3fea2e63 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:34.598419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab582875-68df-4fd8-9dfe-4fec72de8b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:34.599380 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab582875-68df-4fd8-9dfe-4fec72de8b5f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:34.599886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab582875-68df-4fd8-9dfe-4fec72de8b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:34.600283 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab582875-68df-4fd8-9dfe-4fec72de8b5f None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:34.600922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab582875-68df-4fd8-9dfe-4fec72de8b5f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:34.601226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab582875-68df-4fd8-9dfe-4fec72de8b5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:34.602735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab582875-68df-4fd8-9dfe-4fec72de8b5f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:34.620172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 439/841] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:20:34 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:34.620172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab582875-68df-4fd8-9dfe-4fec72de8b5f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:34.660717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44161ee-a9f0-4b27-a477-ded0e5a3b6fe None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:34.684800 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:34.710639 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3643bd-bd2a-468c-a66b-103e63c08318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:34.711363 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3643bd-bd2a-468c-a66b-103e63c08318 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:34.711716 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3643bd-bd2a-468c-a66b-103e63c08318 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:34.712098 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3643bd-bd2a-468c-a66b-103e63c08318 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:34.712887 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3643bd-bd2a-468c-a66b-103e63c08318 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:34.713259 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3643bd-bd2a-468c-a66b-103e63c08318 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:34.714827 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3643bd-bd2a-468c-a66b-103e63c08318 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:34.729662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 403/842] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:34.730278 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:34.737269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3643bd-bd2a-468c-a66b-103e63c08318 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:34.767739 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab582875-68df-4fd8-9dfe-4fec72de8b5f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:34.809568 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd201d6a-de29-4b27-9f3c-de7d2a951855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:34.810804 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd201d6a-de29-4b27-9f3c-de7d2a951855 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:34.810997 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd201d6a-de29-4b27-9f3c-de7d2a951855 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:34.811323 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd201d6a-de29-4b27-9f3c-de7d2a951855 None admin] PATH_INFO: `/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/61709ccc1e244185b252179451cd860f/roles/04bc48cc5b6842029b93a845d6744985` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:34.828853 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd201d6a-de29-4b27-9f3c-de7d2a951855 None admin] RBAC: Authorizing `identity:create_grant(project_id=90b25dc95d5a4a068994bdd801d6b8ea, user_id=61709ccc1e244185b252179451cd860f, role_id=04bc48cc5b6842029b93a845d6744985)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:34.830371 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd201d6a-de29-4b27-9f3c-de7d2a951855 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:34.849710 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 440/843] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:20:34 2022] PUT /identity/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/61709ccc1e244185b252179451cd860f/roles/04bc48cc5b6842029b93a845d6744985 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:20:34.888274 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3643bd-bd2a-468c-a66b-103e63c08318 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:34.930327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffaff1-0732-4362-9299-74cecfd85a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:34.930985 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffaff1-0732-4362-9299-74cecfd85a62 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:34.931263 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffaff1-0732-4362-9299-74cecfd85a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:34.931536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ffaff1-0732-4362-9299-74cecfd85a62 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:34.941754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffaff1-0732-4362-9299-74cecfd85a62 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:34.943452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ffaff1-0732-4362-9299-74cecfd85a62 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:34.958536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-19ffaff1-0732-4362-9299-74cecfd85a62 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:20:34.958536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:34.958536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:34.958536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:34.958536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:34.958536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:34.958536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:34.958536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:34.958536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:34.958536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:34.958536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:34.958536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:34.958536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:34.958536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:34.958536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:34.958536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:34.959582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:34.959582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:34.959582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:34.959582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:34.959582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:34.959582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:34.959582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:34.959582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:34.959582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:34.959582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:34.959582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:34.959582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:34.959582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:34.959582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:34.959582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:34.959582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:34.959582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:34.959582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:34.960604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:34.960604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:34.960604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:34.960604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:34.960604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:34.960604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:34.960604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:34.960604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:34.960604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 29 20:20:34.960604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:34.960604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 404/844] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:20:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:34.967282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd201d6a-de29-4b27-9f3c-de7d2a951855 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:35.105644 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:35.113998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd201d6a-de29-4b27-9f3c-de7d2a951855 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:35.153592 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73deeede-8058-4049-8441-31b10d03dbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:35.154210 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73deeede-8058-4049-8441-31b10d03dbef None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:35.154547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73deeede-8058-4049-8441-31b10d03dbef None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:35.154816 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73deeede-8058-4049-8441-31b10d03dbef None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:35.155602 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73deeede-8058-4049-8441-31b10d03dbef None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:35.155908 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73deeede-8058-4049-8441-31b10d03dbef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:35.157563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73deeede-8058-4049-8441-31b10d03dbef None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:35.184216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 441/845] 10.222.0.18 () {56 vars in 1135 bytes} [Mon Aug 29 20:20:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:35.190145 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffaff1-0732-4362-9299-74cecfd85a62 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:35.251321 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:35.308204 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ffaff1-0732-4362-9299-74cecfd85a62 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:35.343594 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7561eb4-6ae4-468b-a0e0-51edda392c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:35.344233 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7561eb4-6ae4-468b-a0e0-51edda392c1b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:35.344512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7561eb4-6ae4-468b-a0e0-51edda392c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:35.344782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7561eb4-6ae4-468b-a0e0-51edda392c1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:35.427308 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7561eb4-6ae4-468b-a0e0-51edda392c1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:35.428753 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7561eb4-6ae4-468b-a0e0-51edda392c1b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:35.494501 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:35.563107 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 405/846] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:20:35 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:20:35.568704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73deeede-8058-4049-8441-31b10d03dbef None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:35.683588 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73deeede-8058-4049-8441-31b10d03dbef None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:35.719901 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d830e80-3d08-4bf4-a782-03afecefb9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:35.720566 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d830e80-3d08-4bf4-a782-03afecefb9df None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:35.720845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d830e80-3d08-4bf4-a782-03afecefb9df None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:35.721117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d830e80-3d08-4bf4-a782-03afecefb9df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:35.733011 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d830e80-3d08-4bf4-a782-03afecefb9df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:35.734534 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d830e80-3d08-4bf4-a782-03afecefb9df None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:35.751253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-9d830e80-3d08-4bf4-a782-03afecefb9df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:35.751253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:35.751253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:35.751253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:35.751253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:35.751253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:35.751253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:35.751253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:35.751253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:35.751253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:35.751253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:35.751253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:35.751253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:35.751253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:35.751253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:35.751253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:35.752277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:35.752277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:35.752277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:35.752277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:35.752277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:35.752277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:35.752277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:35.752277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:35.752277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:35.752277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:35.752277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:35.752277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:35.752277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:35.752277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:35.752277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:35.752277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:35.752277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:35.754643 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:35.754643 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:35.754643 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:35.754643 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 442/847] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:35.758186 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7561eb4-6ae4-468b-a0e0-51edda392c1b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:35.881004 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7561eb4-6ae4-468b-a0e0-51edda392c1b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:35.906220 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3235ce-7666-4d6f-8376-f9dc7ee2fb49 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:35.906551 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3235ce-7666-4d6f-8376-f9dc7ee2fb49 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:35.906774 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3235ce-7666-4d6f-8376-f9dc7ee2fb49 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:35.908392 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 443/848] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:35.913184 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62608702-0e31-49ca-9c6d-c3de953fd4f6 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:35.913471 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62608702-0e31-49ca-9c6d-c3de953fd4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:35.913750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62608702-0e31-49ca-9c6d-c3de953fd4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:35.922013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:35.925606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999cbdc-0758-47bd-92e0-ac7fd690bdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:35.926765 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999cbdc-0758-47bd-92e0-ac7fd690bdc5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:35.927189 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999cbdc-0758-47bd-92e0-ac7fd690bdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:35.927604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999cbdc-0758-47bd-92e0-ac7fd690bdc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:35.928513 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999cbdc-0758-47bd-92e0-ac7fd690bdc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:35.928961 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999cbdc-0758-47bd-92e0-ac7fd690bdc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:35.930504 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999cbdc-0758-47bd-92e0-ac7fd690bdc5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:35.934440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-62608702-0e31-49ca-9c6d-c3de953fd4f6 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:35.935698 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-62608702-0e31-49ca-9c6d-c3de953fd4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:35.943805 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 406/849] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:35.949643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999cbdc-0758-47bd-92e0-ac7fd690bdc5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:35.968926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:36.044396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 444/850] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:20:36.070887 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999cbdc-0758-47bd-92e0-ac7fd690bdc5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:36.108651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7a4af-4e31-451b-a6fc-83cfa429d1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:36.109292 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7a4af-4e31-451b-a6fc-83cfa429d1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:36.109570 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7a4af-4e31-451b-a6fc-83cfa429d1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:36.109899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7a4af-4e31-451b-a6fc-83cfa429d1e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:36.117134 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7a4af-4e31-451b-a6fc-83cfa429d1e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:36.118866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7a4af-4e31-451b-a6fc-83cfa429d1e9 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:36.131010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-4db7a4af-4e31-451b-a6fc-83cfa429d1e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:36.131010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:36.131010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:36.131010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:36.131010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:36.131010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:36.131010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:36.131010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:36.131010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:36.131010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:36.131010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:36.131010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:36.131010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:36.131010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:36.131010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:36.131920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:36.131920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:36.131920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:36.131920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:36.131920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:36.131920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:36.131920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:36.131920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:36.131920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:36.131920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:36.131920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:36.131920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:36.131920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:36.131920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:36.131920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:36.131920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:36.131920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:36.132834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:36.132834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:36.132834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:36.132834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:36.132834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:36.132834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:36.132834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:36.132834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:36.132834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 407/851] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:36.139082 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62608702-0e31-49ca-9c6d-c3de953fd4f6 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:36.252789 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62608702-0e31-49ca-9c6d-c3de953fd4f6 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:36.291322 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275199c-20fc-4a75-aba8-bc2ee3d55362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:36.291973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275199c-20fc-4a75-aba8-bc2ee3d55362 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:36.292310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275199c-20fc-4a75-aba8-bc2ee3d55362 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:36.292611 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275199c-20fc-4a75-aba8-bc2ee3d55362 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:36.293330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275199c-20fc-4a75-aba8-bc2ee3d55362 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:36.293639 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275199c-20fc-4a75-aba8-bc2ee3d55362 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:36.299984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275199c-20fc-4a75-aba8-bc2ee3d55362 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:36.299984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:36.310239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 445/852] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:36.316350 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7a4af-4e31-451b-a6fc-83cfa429d1e9 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:36.328619 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275199c-20fc-4a75-aba8-bc2ee3d55362 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:36.402074 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:36.442839 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7a4af-4e31-451b-a6fc-83cfa429d1e9 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:36.461957 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275199c-20fc-4a75-aba8-bc2ee3d55362 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:36.484250 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef053b-46f9-4ccb-8c05-c9a8d33ebb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:36.484873 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef053b-46f9-4ccb-8c05-c9a8d33ebb18 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:36.485155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef053b-46f9-4ccb-8c05-c9a8d33ebb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:36.485429 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef053b-46f9-4ccb-8c05-c9a8d33ebb18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:36.486429 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef053b-46f9-4ccb-8c05-c9a8d33ebb18 None admin] RBAC: Adding query filter params (role.id=322da7bd64024770a64bdb73e5c54719, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=599129b7d8ff47efb145787414ddb80d) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:36.488453 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef053b-46f9-4ccb-8c05-c9a8d33ebb18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:36.489921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef053b-46f9-4ccb-8c05-c9a8d33ebb18 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:36.505884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 408/853] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:20:36 2022] GET /identity/v3/role_assignments?user.id=599129b7d8ff47efb145787414ddb80d&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=322da7bd64024770a64bdb73e5c54719 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:36.509332 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a626a92-36cd-4f89-9ced-7a6c45644315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:36.510348 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a626a92-36cd-4f89-9ced-7a6c45644315 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:36.510839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a626a92-36cd-4f89-9ced-7a6c45644315 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:36.511417 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a626a92-36cd-4f89-9ced-7a6c45644315 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:36.512348 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a626a92-36cd-4f89-9ced-7a6c45644315 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:36.514024 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a626a92-36cd-4f89-9ced-7a6c45644315 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:36.524678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-5a626a92-36cd-4f89-9ced-7a6c45644315 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:20:36.524678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:36.524678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:36.524678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:36.524678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:36.524678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:36.524678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:36.524678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:36.524678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:20:36.524678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:36.524678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:36.524678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:36.524678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:36.524678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:36.524678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:36.524678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:36.525664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:36.525664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:36.525664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:36.525664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:36.525664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:36.525664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:36.525664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:36.525664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:36.525664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:36.525664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:36.525664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:36.525664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:36.525664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:36.525664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:36.525664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:36.525664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:36.525664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:36.526561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:36.526561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:36.526561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:36.526561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:36.526561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:36.526561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:36.526561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:36.526561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:36.526561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:36.526561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:36.526561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:36.526561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:36.526561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:20:36.526561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:36.530150 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 446/854] 10.222.0.18 () {56 vars in 1132 bytes} [Mon Aug 29 20:20:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:36.536578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef053b-46f9-4ccb-8c05-c9a8d33ebb18 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:36.658329 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef053b-46f9-4ccb-8c05-c9a8d33ebb18 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:36.696510 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463a7b8b-cda1-412a-8cf4-b1a6de30d79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:36.697120 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a7b8b-cda1-412a-8cf4-b1a6de30d79c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:36.697396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a7b8b-cda1-412a-8cf4-b1a6de30d79c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:36.697665 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463a7b8b-cda1-412a-8cf4-b1a6de30d79c None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:36.698298 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463a7b8b-cda1-412a-8cf4-b1a6de30d79c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:36.698649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463a7b8b-cda1-412a-8cf4-b1a6de30d79c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:36.700358 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463a7b8b-cda1-412a-8cf4-b1a6de30d79c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:36.701691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:36.706238 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 409/855] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 29 20:20:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:36.711801 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a626a92-36cd-4f89-9ced-7a6c45644315 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:36.762366 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01feb4-98d1-4c5f-b89f-78e2515185e4 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:36.762558 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01feb4-98d1-4c5f-b89f-78e2515185e4 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:36.762831 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01feb4-98d1-4c5f-b89f-78e2515185e4 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:36.765208 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 410/856] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:20:36.771392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bdd63-11e0-4eda-9655-85ca85468f6d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:36.771392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bdd63-11e0-4eda-9655-85ca85468f6d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:36.771814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bdd63-11e0-4eda-9655-85ca85468f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:36.796199 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:36.799896 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-553bdd63-11e0-4eda-9655-85ca85468f6d None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:36.801652 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-553bdd63-11e0-4eda-9655-85ca85468f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:36.851509 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a626a92-36cd-4f89-9ced-7a6c45644315 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:36.892014 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 411/857] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:36.895185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202896e-e9e4-4368-a095-d0f72997c5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:36.895865 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202896e-e9e4-4368-a095-d0f72997c5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:36.896205 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202896e-e9e4-4368-a095-d0f72997c5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:36.896450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202896e-e9e4-4368-a095-d0f72997c5b7 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:36.897089 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202896e-e9e4-4368-a095-d0f72997c5b7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:36.897394 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202896e-e9e4-4368-a095-d0f72997c5b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:36.898751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202896e-e9e4-4368-a095-d0f72997c5b7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:36.903943 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 447/858] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 29 20:20:36 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:37.015603 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0c891-6e64-4004-b05a-ac753155e1df None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:37.016140 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0c891-6e64-4004-b05a-ac753155e1df None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:37.016140 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0c891-6e64-4004-b05a-ac753155e1df None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:37.017831 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 412/859] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:37.023530 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2f68d-cf71-4aea-8432-3e57564b21ed None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:37.023822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2f68d-cf71-4aea-8432-3e57564b21ed None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:37.024125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de2f68d-cf71-4aea-8432-3e57564b21ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:37.044651 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4de2f68d-cf71-4aea-8432-3e57564b21ed None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:37.045866 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4de2f68d-cf71-4aea-8432-3e57564b21ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:37.125378 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 448/860] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:37.159159 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0c891-6e64-4004-b05a-ac753155e1df None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:37.268463 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0c891-6e64-4004-b05a-ac753155e1df None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:37.305857 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:37.315241 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8612d-4f37-43a3-b2cd-ce262652e570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:37.315857 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8612d-4f37-43a3-b2cd-ce262652e570 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:37.316165 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8612d-4f37-43a3-b2cd-ce262652e570 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:37.316436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8612d-4f37-43a3-b2cd-ce262652e570 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:37.317110 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8612d-4f37-43a3-b2cd-ce262652e570 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:37.318523 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8612d-4f37-43a3-b2cd-ce262652e570 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:37.335743 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 413/861] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 29 20:20:37 2022] POST /identity/v3/services => generated 251 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:20:37.397175 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2f68d-cf71-4aea-8432-3e57564b21ed None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:37.454511 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:37.507600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de2f68d-cf71-4aea-8432-3e57564b21ed None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:37.552621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c0482-d120-4b9a-b676-cbc6b24ffea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:37.553269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c0482-d120-4b9a-b676-cbc6b24ffea0 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:37.553546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c0482-d120-4b9a-b676-cbc6b24ffea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:37.553812 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c0482-d120-4b9a-b676-cbc6b24ffea0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:37.564562 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c0482-d120-4b9a-b676-cbc6b24ffea0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:37.569411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-b01c0482-d120-4b9a-b676-cbc6b24ffea0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:37.569411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:37.569411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:37.569411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:37.569411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:37.569411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:37.569411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:37.569411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:37.569411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:37.569411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:37.569411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:37.569411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:37.569411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:37.569411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:37.569411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:37.569411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:37.570327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:37.570327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:37.570327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:37.570327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:37.570327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:37.570327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:37.570327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:37.570327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:37.570327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:37.570327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:37.570327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:37.570327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:37.570327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:37.570327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:37.570327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:37.570327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:37.571202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:37.571202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:37.571202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:37.571202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:37.571202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:37.571202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:37.571202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:37.571202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:37.571202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:37.571202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:37.571202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:20:37.571202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:37.571202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 449/862] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:20:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:37.577080 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8612d-4f37-43a3-b2cd-ce262652e570 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:37.684593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8612d-4f37-43a3-b2cd-ce262652e570 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:37.724860 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a77624-e1d8-4318-8e5e-6275a4217117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:37.725474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a77624-e1d8-4318-8e5e-6275a4217117 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:37.725756 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a77624-e1d8-4318-8e5e-6275a4217117 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:37.726031 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a77624-e1d8-4318-8e5e-6275a4217117 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:37.726702 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a77624-e1d8-4318-8e5e-6275a4217117 None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:37.727007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a77624-e1d8-4318-8e5e-6275a4217117 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:37.728418 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a77624-e1d8-4318-8e5e-6275a4217117 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:37.737309 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 414/863] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:20:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:37.742638 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c0482-d120-4b9a-b676-cbc6b24ffea0 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:37.849071 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c0482-d120-4b9a-b676-cbc6b24ffea0 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:37.860704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:37.889092 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a91b35-170a-475f-9f00-d505b261919d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:37.890591 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a91b35-170a-475f-9f00-d505b261919d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:37.891354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a91b35-170a-475f-9f00-d505b261919d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:37.891861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a91b35-170a-475f-9f00-d505b261919d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:37.904421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a91b35-170a-475f-9f00-d505b261919d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:37.906159 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a91b35-170a-475f-9f00-d505b261919d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:37.922052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-71a91b35-170a-475f-9f00-d505b261919d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:37.922052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:37.922052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:37.922052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:37.922052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:37.922052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:37.922052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:37.922052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:37.922052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:37.922052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:37.922052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:37.922052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:37.922052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:37.922052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:37.922052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:37.922052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:37.923088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:37.923088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:37.923088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:37.923088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:37.923088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:37.923088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:37.923088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:37.923088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:37.923088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:37.923088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:37.923088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:37.923088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:37.923088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:37.923088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:37.923088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:37.923088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:37.923088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:37.923984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:37.923984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:37.923984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:37.926041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 450/864] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:37.933245 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a77624-e1d8-4318-8e5e-6275a4217117 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:37.994891 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:38.049612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a77624-e1d8-4318-8e5e-6275a4217117 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:38.088394 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc141b6-cb32-4e5c-960a-b54c8337d1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:38.089366 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc141b6-cb32-4e5c-960a-b54c8337d1b1 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:38.089866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc141b6-cb32-4e5c-960a-b54c8337d1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:38.090341 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc141b6-cb32-4e5c-960a-b54c8337d1b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:38.091173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc141b6-cb32-4e5c-960a-b54c8337d1b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:38.091674 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc141b6-cb32-4e5c-960a-b54c8337d1b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:38.093270 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc141b6-cb32-4e5c-960a-b54c8337d1b1 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:38.106091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 415/865] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:38.113112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a91b35-170a-475f-9f00-d505b261919d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:38.220229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a91b35-170a-475f-9f00-d505b261919d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:38.255146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6896903-cdfb-4fec-8160-1c9e1cbcf974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:38.255802 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6896903-cdfb-4fec-8160-1c9e1cbcf974 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:38.256110 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6896903-cdfb-4fec-8160-1c9e1cbcf974 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:38.256383 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6896903-cdfb-4fec-8160-1c9e1cbcf974 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:38.265569 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6896903-cdfb-4fec-8160-1c9e1cbcf974 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:38.267133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6896903-cdfb-4fec-8160-1c9e1cbcf974 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:38.279520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-a6896903-cdfb-4fec-8160-1c9e1cbcf974 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:20:38.279520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:38.279520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:38.279520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:38.279520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:38.279520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:38.279520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:38.279520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:38.279520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:38.279520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:38.279520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:38.279520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:38.279520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:38.279520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:38.279520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:38.279520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:38.280546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:38.280546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:38.280546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:38.280546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:38.280546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:38.280546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:38.280546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:38.280546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:38.280546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:38.280546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:38.280546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:38.280546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:38.280546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:38.280546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:38.280546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:38.280546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:38.280546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:38.280546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:38.281481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:38.281481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:38.281481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:38.281481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:38.281481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:38.281481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:38.281481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:38.281481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:38.281481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:20:38.281481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:38.281481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 451/866] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Aug 29 20:20:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:38.288543 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc141b6-cb32-4e5c-960a-b54c8337d1b1 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:38.415456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc141b6-cb32-4e5c-960a-b54c8337d1b1 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:38.431658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:38.460404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90dcd9b-33bb-4edc-942b-f35b6a16d960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:38.461294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dcd9b-33bb-4edc-942b-f35b6a16d960 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:38.461720 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dcd9b-33bb-4edc-942b-f35b6a16d960 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:38.462138 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dcd9b-33bb-4edc-942b-f35b6a16d960 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:38.463023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90dcd9b-33bb-4edc-942b-f35b6a16d960 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:38.463473 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90dcd9b-33bb-4edc-942b-f35b6a16d960 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:38.465645 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90dcd9b-33bb-4edc-942b-f35b6a16d960 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:38.484880 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 416/867] 10.222.0.18 () {56 vars in 1149 bytes} [Mon Aug 29 20:20:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:38.493194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6896903-cdfb-4fec-8160-1c9e1cbcf974 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:38.536353 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:38.601562 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6896903-cdfb-4fec-8160-1c9e1cbcf974 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:38.641329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75d36a-0888-4a57-9181-de03c53c7702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:38.642333 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75d36a-0888-4a57-9181-de03c53c7702 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:38.642813 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75d36a-0888-4a57-9181-de03c53c7702 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:38.643298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75d36a-0888-4a57-9181-de03c53c7702 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:38.702093 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282e64e-f267-446a-b26c-16efb5479fba None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:38.702293 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282e64e-f267-446a-b26c-16efb5479fba None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:38.702629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282e64e-f267-446a-b26c-16efb5479fba None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:38.704230 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 417/868] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:20:38.709028 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71397f-3d3c-4542-8942-6a3d43e3f002 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:38.709308 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71397f-3d3c-4542-8942-6a3d43e3f002 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:38.709575 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71397f-3d3c-4542-8942-6a3d43e3f002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:38.719619 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75d36a-0888-4a57-9181-de03c53c7702 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:38.721359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75d36a-0888-4a57-9181-de03c53c7702 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:38.731080 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-4a71397f-3d3c-4542-8942-6a3d43e3f002 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:38.732320 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4a71397f-3d3c-4542-8942-6a3d43e3f002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:38.744813 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:38.828043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 418/869] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:38 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:38.839597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 452/870] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:20:38 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:20:38.849024 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71397f-3d3c-4542-8942-6a3d43e3f002 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:38.946078 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:38.960031 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71397f-3d3c-4542-8942-6a3d43e3f002 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:38.991760 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd6353-08e0-4c18-afa1-1768d62bb9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:38.992426 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd6353-08e0-4c18-afa1-1768d62bb9f3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:38.992698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd6353-08e0-4c18-afa1-1768d62bb9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:38.992969 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd6353-08e0-4c18-afa1-1768d62bb9f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:39.004174 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd6353-08e0-4c18-afa1-1768d62bb9f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:39.005627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd6353-08e0-4c18-afa1-1768d62bb9f3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:39.021427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-f4bd6353-08e0-4c18-afa1-1768d62bb9f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:39.021427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:39.021427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:39.021427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:39.021427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:39.021427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:39.021427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:39.021427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:39.021427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:39.021427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:39.021427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:39.021427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:39.021427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:39.021427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:39.021427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:39.021427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:39.022395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:39.022395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:39.022395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:39.022395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:39.022395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:39.022395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:39.022395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:39.022395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:39.022395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:39.022395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:39.022395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:39.022395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:39.022395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:39.022395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:39.022395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:39.022395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:39.022395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:39.023440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:39.023440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:39.023440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:39.023440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 419/871] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:39.028461 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75d36a-0888-4a57-9181-de03c53c7702 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:39.046529 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b26169-e6f4-4b77-88b2-9944faf28a83 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:39.046833 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b26169-e6f4-4b77-88b2-9944faf28a83 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:39.047072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b26169-e6f4-4b77-88b2-9944faf28a83 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:39.048730 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 420/872] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:20:39.053677 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82087d67-e664-4d11-9491-6875ebd23f2f None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:39.054328 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82087d67-e664-4d11-9491-6875ebd23f2f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:39.054328 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82087d67-e664-4d11-9491-6875ebd23f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:39.074561 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-82087d67-e664-4d11-9491-6875ebd23f2f None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:39.075750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-82087d67-e664-4d11-9491-6875ebd23f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:39.137000 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75d36a-0888-4a57-9181-de03c53c7702 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:39.162555 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:39.171974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 421/873] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:39 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:20:39.172827 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82087d67-e664-4d11-9491-6875ebd23f2f None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:39.183526 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162ff52-e8c2-432e-a2b3-b364ebc9fd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:39.184452 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162ff52-e8c2-432e-a2b3-b364ebc9fd14 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:39.186820 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162ff52-e8c2-432e-a2b3-b364ebc9fd14 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:39.189305 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162ff52-e8c2-432e-a2b3-b364ebc9fd14 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:39.193855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162ff52-e8c2-432e-a2b3-b364ebc9fd14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:39.195542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162ff52-e8c2-432e-a2b3-b364ebc9fd14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:39.201127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162ff52-e8c2-432e-a2b3-b364ebc9fd14 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:39.219520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 453/874] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:39.228067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162ff52-e8c2-432e-a2b3-b364ebc9fd14 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:39.266418 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:39.289742 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82087d67-e664-4d11-9491-6875ebd23f2f None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:39.325142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66c20d-c4ba-430c-b906-2d30c7e11e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:39.325778 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66c20d-c4ba-430c-b906-2d30c7e11e72 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:39.326053 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66c20d-c4ba-430c-b906-2d30c7e11e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:39.326323 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66c20d-c4ba-430c-b906-2d30c7e11e72 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:39.326907 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66c20d-c4ba-430c-b906-2d30c7e11e72 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:39.328308 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66c20d-c4ba-430c-b906-2d30c7e11e72 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:39.337597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-8b66c20d-c4ba-430c-b906-2d30c7e11e72 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:20:39.337597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:39.337597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:39.337597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:39.337597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:39.337597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:39.337597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:39.337597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:39.337597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:20:39.337597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:39.337597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:39.337597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:39.337597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:39.337597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:39.337597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:39.337597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:39.337597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:39.338798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:39.338798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:39.338798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:39.338798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:39.338798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:39.338798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:39.338798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:39.338798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:39.338798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:39.338798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:39.338798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:39.338798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:39.338798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:39.338798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:39.338798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:39.338798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:39.340427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:39.340427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:39.340427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:39.340427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:39.340427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:39.340427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:39.340427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:39.340427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:39.340427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:39.340427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:39.340427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:39.340427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:39.340427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:20:39.340427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:39.340427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 422/875] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 29 20:20:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:39.347924 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162ff52-e8c2-432e-a2b3-b364ebc9fd14 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:39.356305 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66c20d-c4ba-430c-b906-2d30c7e11e72 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:39.390551 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096fb97-0721-4099-a199-d6bdfa2b3793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:39.391885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096fb97-0721-4099-a199-d6bdfa2b3793 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:39.392688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096fb97-0721-4099-a199-d6bdfa2b3793 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:39.393614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096fb97-0721-4099-a199-d6bdfa2b3793 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:39.404492 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096fb97-0721-4099-a199-d6bdfa2b3793 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:39.406366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096fb97-0721-4099-a199-d6bdfa2b3793 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:39.423540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-a096fb97-0721-4099-a199-d6bdfa2b3793 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:39.423540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:39.423540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:39.423540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:39.423540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:39.423540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:39.423540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:39.423540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:39.423540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:39.423540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:39.423540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:39.423540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:39.423540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:39.423540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:39.423540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:39.424513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:39.424513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:39.424513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:39.424513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:39.424513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:39.424513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:39.424513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:39.424513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:39.424513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:39.424513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:39.424513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:39.424513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:39.424513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:39.424513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:39.424513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:39.424513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:39.424513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:39.425412 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:39.425412 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:39.425412 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:39.425412 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:39.425412 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:39.425412 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:39.425412 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:39.425412 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:39.428778 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 454/876] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:39.434983 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096fb97-0721-4099-a199-d6bdfa2b3793 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:39.474658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66c20d-c4ba-430c-b906-2d30c7e11e72 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:39.488429 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:39.510317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a8089-f965-470f-8f28-a1ae1cab3c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:39.510929 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a8089-f965-470f-8f28-a1ae1cab3c7d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:39.511228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a8089-f965-470f-8f28-a1ae1cab3c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:39.511506 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a8089-f965-470f-8f28-a1ae1cab3c7d None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:39.512171 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a8089-f965-470f-8f28-a1ae1cab3c7d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:39.512468 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a8089-f965-470f-8f28-a1ae1cab3c7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:39.513838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a8089-f965-470f-8f28-a1ae1cab3c7d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:39.519362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 423/877] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Aug 29 20:20:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:39.520153 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a8089-f965-470f-8f28-a1ae1cab3c7d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:39.531928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:39.549132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096fb97-0721-4099-a199-d6bdfa2b3793 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:39.584823 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb4fa2-6da7-49e6-ae1f-ced2b8a3566f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:39.585432 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb4fa2-6da7-49e6-ae1f-ced2b8a3566f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:39.585713 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb4fa2-6da7-49e6-ae1f-ced2b8a3566f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:39.585987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb4fa2-6da7-49e6-ae1f-ced2b8a3566f None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:39.586682 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb4fa2-6da7-49e6-ae1f-ced2b8a3566f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:39.586989 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb4fa2-6da7-49e6-ae1f-ced2b8a3566f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:39.588494 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb4fa2-6da7-49e6-ae1f-ced2b8a3566f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:39.601585 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 455/878] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:39.602797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938140d-e155-43c3-b5b4-b7ff87bdf236 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:39.603084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938140d-e155-43c3-b5b4-b7ff87bdf236 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:39.603357 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938140d-e155-43c3-b5b4-b7ff87bdf236 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:39.604736 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 456/879] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:39.605956 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938140d-e155-43c3-b5b4-b7ff87bdf236 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:39.627195 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a8089-f965-470f-8f28-a1ae1cab3c7d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:39.662250 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a6cee-06a2-4d07-a325-2deb1cf07089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:39.662856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a6cee-06a2-4d07-a325-2deb1cf07089 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:39.663131 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a6cee-06a2-4d07-a325-2deb1cf07089 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:39.663398 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a6cee-06a2-4d07-a325-2deb1cf07089 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:39.664322 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a6cee-06a2-4d07-a325-2deb1cf07089 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:39.665672 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a6cee-06a2-4d07-a325-2deb1cf07089 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:39.680665 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 424/880] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 29 20:20:39 2022] POST /identity/v3/services => generated 269 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:20:39.681696 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a6cee-06a2-4d07-a325-2deb1cf07089 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:39.719292 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:39.731658 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938140d-e155-43c3-b5b4-b7ff87bdf236 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:39.758354 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:39.771610 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6d789-c989-451e-8083-64810b8f4d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:39.773052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6d789-c989-451e-8083-64810b8f4d57 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:39.773052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6d789-c989-451e-8083-64810b8f4d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:39.773052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6d789-c989-451e-8083-64810b8f4d57 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:39.773600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6d789-c989-451e-8083-64810b8f4d57 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:39.773691 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6d789-c989-451e-8083-64810b8f4d57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:39.775082 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6d789-c989-451e-8083-64810b8f4d57 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:39.780829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 457/881] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Aug 29 20:20:39 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:39.781938 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7631d7-b7c9-4fc4-ab37-e45fa07f81c2 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:39.782249 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7631d7-b7c9-4fc4-ab37-e45fa07f81c2 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:39.782503 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7631d7-b7c9-4fc4-ab37-e45fa07f81c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:39.807028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-bd7631d7-b7c9-4fc4-ab37-e45fa07f81c2 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:39.808885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-bd7631d7-b7c9-4fc4-ab37-e45fa07f81c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:39.818901 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a6cee-06a2-4d07-a325-2deb1cf07089 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:39.867684 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0388d-a335-4227-8567-792655199a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:39.868373 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0388d-a335-4227-8567-792655199a85 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:39.868652 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0388d-a335-4227-8567-792655199a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:39.868923 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0388d-a335-4227-8567-792655199a85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:39.869681 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0388d-a335-4227-8567-792655199a85 None admin] RBAC: Adding query filter params (role.id=04bc48cc5b6842029b93a845d6744985, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=61709ccc1e244185b252179451cd860f) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:39.869970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0388d-a335-4227-8567-792655199a85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:39.871466 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d0388d-a335-4227-8567-792655199a85 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:39.884392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 425/882] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:20:39 2022] GET /identity/v3/role_assignments?user.id=61709ccc1e244185b252179451cd860f&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=04bc48cc5b6842029b93a845d6744985 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:39.910937 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 458/883] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:40.187176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0388d-a335-4227-8567-792655199a85 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:40.298246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0388d-a335-4227-8567-792655199a85 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:40.329211 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:40.334879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e74ef-030e-46e0-a9f3-08a4d1d60aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:40.335596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e74ef-030e-46e0-a9f3-08a4d1d60aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:40.335966 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e74ef-030e-46e0-a9f3-08a4d1d60aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:40.336315 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e74ef-030e-46e0-a9f3-08a4d1d60aa8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:40.336969 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e74ef-030e-46e0-a9f3-08a4d1d60aa8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:40.338928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e74ef-030e-46e0-a9f3-08a4d1d60aa8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:40.348307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-335e74ef-030e-46e0-a9f3-08a4d1d60aa8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:20:40.348307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:40.348307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:40.348307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:40.348307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:40.348307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:40.348307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:40.348307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:40.348307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:20:40.348307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:40.348307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:40.348307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:40.348307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:40.348307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:40.348307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:40.348307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:40.348307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:40.349518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:40.349518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:40.349518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:40.349518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:40.349518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:40.349518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:40.349518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:40.349518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:40.349518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:40.349518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:40.349518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:40.349518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:40.349518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:40.349518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:40.349518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:40.349518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:40.350548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:40.350548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:40.350548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:40.350548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:40.350548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:40.350548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:40.350548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:40.350548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:40.350548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:40.350548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:40.350548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:40.350548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:40.350548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:20:40.350548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:40.352807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 426/884] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:20:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:40.357951 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7631d7-b7c9-4fc4-ab37-e45fa07f81c2 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:40.458733 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:40.468724 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7631d7-b7c9-4fc4-ab37-e45fa07f81c2 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:40.505229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4391fe1-9f1e-4070-8446-0f07adbc4f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:40.505854 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4391fe1-9f1e-4070-8446-0f07adbc4f8c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:40.506137 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4391fe1-9f1e-4070-8446-0f07adbc4f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:40.506415 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4391fe1-9f1e-4070-8446-0f07adbc4f8c None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:40.507061 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4391fe1-9f1e-4070-8446-0f07adbc4f8c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:40.507367 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4391fe1-9f1e-4070-8446-0f07adbc4f8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:40.508777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4391fe1-9f1e-4070-8446-0f07adbc4f8c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:40.521910 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 459/885] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:20:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:40.523289 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e74ef-030e-46e0-a9f3-08a4d1d60aa8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:40.629760 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e74ef-030e-46e0-a9f3-08a4d1d60aa8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:40.666101 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267fe29-025f-44f6-a9df-ce1cc2cc2c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:40.666803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267fe29-025f-44f6-a9df-ce1cc2cc2c74 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:40.667084 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267fe29-025f-44f6-a9df-ce1cc2cc2c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:40.667357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5267fe29-025f-44f6-a9df-ce1cc2cc2c74 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:40.668138 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5267fe29-025f-44f6-a9df-ce1cc2cc2c74 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:40.668446 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5267fe29-025f-44f6-a9df-ce1cc2cc2c74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:40.669891 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5267fe29-025f-44f6-a9df-ce1cc2cc2c74 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:40.680151 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 427/886] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:20:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:40.681638 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4391fe1-9f1e-4070-8446-0f07adbc4f8c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:40.792999 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4391fe1-9f1e-4070-8446-0f07adbc4f8c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:40.830709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29ca84-16ee-4870-a6ab-2ed3b3e31dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:40.831328 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29ca84-16ee-4870-a6ab-2ed3b3e31dac None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:40.831666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29ca84-16ee-4870-a6ab-2ed3b3e31dac None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:40.831960 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29ca84-16ee-4870-a6ab-2ed3b3e31dac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:40.832685 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29ca84-16ee-4870-a6ab-2ed3b3e31dac None admin] RBAC: Adding query filter params (interface=public, service_id=71ae6dca34474b9abe2ca2e12b6fb720, region_id=RegionOne) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:40.832998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29ca84-16ee-4870-a6ab-2ed3b3e31dac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:40.834374 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29ca84-16ee-4870-a6ab-2ed3b3e31dac None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:40.845113 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 460/887] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Aug 29 20:20:40 2022] GET /identity/v3/endpoints?service_id=71ae6dca34474b9abe2ca2e12b6fb720&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:40.846478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267fe29-025f-44f6-a9df-ce1cc2cc2c74 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:40.917245 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:40.967332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5267fe29-025f-44f6-a9df-ce1cc2cc2c74 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:41.001503 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59630810-e89a-4f46-a5af-368c3bf844fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:41.002187 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59630810-e89a-4f46-a5af-368c3bf844fe None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:41.002517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59630810-e89a-4f46-a5af-368c3bf844fe None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:41.002784 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59630810-e89a-4f46-a5af-368c3bf844fe None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:41.003382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59630810-e89a-4f46-a5af-368c3bf844fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:41.004792 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59630810-e89a-4f46-a5af-368c3bf844fe None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:41.015089 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 428/888] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Aug 29 20:20:40 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:20:41.736509 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae59d0a-a84e-48ba-a611-8950f1ee6b70 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:41.736701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae59d0a-a84e-48ba-a611-8950f1ee6b70 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:41.736980 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae59d0a-a84e-48ba-a611-8950f1ee6b70 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:41.738596 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 461/889] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:41.743805 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd688a-1531-4186-b674-2c921e425f32 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:41.744285 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd688a-1531-4186-b674-2c921e425f32 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:41.744382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd688a-1531-4186-b674-2c921e425f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:41.763625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-f9cd688a-1531-4186-b674-2c921e425f32 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:41.764953 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f9cd688a-1531-4186-b674-2c921e425f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:41.766835 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e9a33-6a76-467e-8f9c-8f1f385b2af6 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:41.767259 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e9a33-6a76-467e-8f9c-8f1f385b2af6 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:41.767668 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e9a33-6a76-467e-8f9c-8f1f385b2af6 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:41.770025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 462/890] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:20:41.776031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f20c1b-ea0f-4540-951f-6b2c1bd4a5de None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:41.776562 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f20c1b-ea0f-4540-951f-6b2c1bd4a5de None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:41.777070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f20c1b-ea0f-4540-951f-6b2c1bd4a5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:41.796074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-77f20c1b-ea0f-4540-951f-6b2c1bd4a5de None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:41.798232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-77f20c1b-ea0f-4540-951f-6b2c1bd4a5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:41.814898 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:41.851214 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 429/891] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:41 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:41.874685 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 463/892] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:41 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:20:42.074510 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8650e8-a21e-4333-b4dd-3d22757d7960 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:42.074794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8650e8-a21e-4333-b4dd-3d22757d7960 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:42.075054 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8650e8-a21e-4333-b4dd-3d22757d7960 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:42.076750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 430/893] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:42.081962 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e34b8-67a9-4e59-a8e7-f966ef523a03 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:42.082254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e34b8-67a9-4e59-a8e7-f966ef523a03 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:42.082535 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e34b8-67a9-4e59-a8e7-f966ef523a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:42.100662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-634e34b8-67a9-4e59-a8e7-f966ef523a03 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:42.101854 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-634e34b8-67a9-4e59-a8e7-f966ef523a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:42.124809 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8650e8-a21e-4333-b4dd-3d22757d7960 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:42.175964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 464/894] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:42 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:42.177544 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e34b8-67a9-4e59-a8e7-f966ef523a03 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:42.193196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:42.249232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8650e8-a21e-4333-b4dd-3d22757d7960 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:42.267409 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:42.289429 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6a188-5467-4efa-b7d6-5571acc89445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:42.290366 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6a188-5467-4efa-b7d6-5571acc89445 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:42.290818 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6a188-5467-4efa-b7d6-5571acc89445 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:42.290976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e34b8-67a9-4e59-a8e7-f966ef523a03 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:42.291377 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6a188-5467-4efa-b7d6-5571acc89445 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:42.292148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6a188-5467-4efa-b7d6-5571acc89445 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:42.294411 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6a188-5467-4efa-b7d6-5571acc89445 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:42.304627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-27b6a188-5467-4efa-b7d6-5571acc89445 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:20:42.304627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:42.304627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:42.304627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:42.304627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:42.304627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:42.304627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:42.304627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:42.304627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:20:42.304627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:42.304627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:42.304627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:42.304627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:42.304627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:42.304627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:42.304627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:42.305593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:42.305593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:42.305593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:42.305593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:42.305593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:42.305593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:42.305593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:42.305593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:42.305593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:42.305593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:42.305593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:42.305593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:42.305593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:42.305593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:42.305593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:42.305593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:42.305593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:42.306478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:42.306478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:42.306478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:42.306478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:42.306478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:42.306478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:42.306478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:42.306478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:42.306478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:42.306478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:42.306478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:42.306478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:42.306478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:20:42.306478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:42.310860 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 431/895] 10.222.0.18 () {56 vars in 1132 bytes} [Mon Aug 29 20:20:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:42.316309 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6a188-5467-4efa-b7d6-5571acc89445 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:42.333606 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f561c-3fdf-42d7-9866-bd0a58d6c4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:42.334125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f561c-3fdf-42d7-9866-bd0a58d6c4c2 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:42.334408 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f561c-3fdf-42d7-9866-bd0a58d6c4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:42.334681 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f561c-3fdf-42d7-9866-bd0a58d6c4c2 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:42.335363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f561c-3fdf-42d7-9866-bd0a58d6c4c2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:42.336791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f561c-3fdf-42d7-9866-bd0a58d6c4c2 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:42.363962 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 465/896] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 29 20:20:42 2022] POST /identity/v3/services => generated 241 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:20:42.443412 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6a188-5467-4efa-b7d6-5571acc89445 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:42.450743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f561c-3fdf-42d7-9866-bd0a58d6c4c2 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:42.469099 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:42.503007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021ec8a-fa4f-4814-bdea-b6ffd7353b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:42.503968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021ec8a-fa4f-4814-bdea-b6ffd7353b2f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:42.504435 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021ec8a-fa4f-4814-bdea-b6ffd7353b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:42.504890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021ec8a-fa4f-4814-bdea-b6ffd7353b2f None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:42.506233 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021ec8a-fa4f-4814-bdea-b6ffd7353b2f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:42.506764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021ec8a-fa4f-4814-bdea-b6ffd7353b2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:42.508767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021ec8a-fa4f-4814-bdea-b6ffd7353b2f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:42.516327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 432/897] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 29 20:20:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:42.522084 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021ec8a-fa4f-4814-bdea-b6ffd7353b2f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:42.585043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f561c-3fdf-42d7-9866-bd0a58d6c4c2 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:42.633662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfed6a3-3880-44fc-9fee-6bfc3bc80359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:42.634473 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfed6a3-3880-44fc-9fee-6bfc3bc80359 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:42.634761 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfed6a3-3880-44fc-9fee-6bfc3bc80359 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:42.635388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfed6a3-3880-44fc-9fee-6bfc3bc80359 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:42.645111 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021ec8a-fa4f-4814-bdea-b6ffd7353b2f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:42.645331 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:42.651512 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfed6a3-3880-44fc-9fee-6bfc3bc80359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:42.658490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-0dfed6a3-3880-44fc-9fee-6bfc3bc80359 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:20:42.658490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:42.658490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:42.658490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:42.658490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:42.658490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:42.658490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:42.658490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:42.658490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:42.658490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:42.658490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:42.658490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:42.658490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:42.658490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:42.658490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:42.658490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:42.659463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:42.659463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:42.659463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:42.659463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:42.659463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:42.659463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:42.659463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:42.659463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:42.659463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:42.659463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:42.659463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:42.659463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:42.659463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:42.659463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:42.659463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:42.659463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:42.660350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:42.660350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:42.660350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:42.660350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:42.660350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:42.660350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:42.660350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:42.660350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:42.660350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:42.660350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:42.660350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:20:42.660350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:42.660350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 466/898] 10.222.0.18 () {56 vars in 1120 bytes} [Mon Aug 29 20:20:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:42.667495 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfed6a3-3880-44fc-9fee-6bfc3bc80359 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:42.690438 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c02a2f-dbdf-4499-85fe-eff39ec5420a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:42.691305 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c02a2f-dbdf-4499-85fe-eff39ec5420a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:42.691737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c02a2f-dbdf-4499-85fe-eff39ec5420a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:42.692187 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c02a2f-dbdf-4499-85fe-eff39ec5420a None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:42.693544 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c02a2f-dbdf-4499-85fe-eff39ec5420a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:42.694005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c02a2f-dbdf-4499-85fe-eff39ec5420a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:42.696094 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c02a2f-dbdf-4499-85fe-eff39ec5420a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:42.704924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 433/899] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 29 20:20:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:42.713000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c02a2f-dbdf-4499-85fe-eff39ec5420a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:42.772986 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:42.797366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfed6a3-3880-44fc-9fee-6bfc3bc80359 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:42.846136 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c02a2f-dbdf-4499-85fe-eff39ec5420a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:42.850550 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062001f-c897-4ec2-93ff-125e254b3470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:42.851235 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062001f-c897-4ec2-93ff-125e254b3470 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:42.851618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062001f-c897-4ec2-93ff-125e254b3470 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:42.851974 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062001f-c897-4ec2-93ff-125e254b3470 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:42.852775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062001f-c897-4ec2-93ff-125e254b3470 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:42.853148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062001f-c897-4ec2-93ff-125e254b3470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:42.854612 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062001f-c897-4ec2-93ff-125e254b3470 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:42.867969 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 467/900] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Aug 29 20:20:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:42.873468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062001f-c897-4ec2-93ff-125e254b3470 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:42.898705 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b15338-a972-43c1-b21b-d8bb680f4580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:42.900043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b15338-a972-43c1-b21b-d8bb680f4580 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:42.900590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b15338-a972-43c1-b21b-d8bb680f4580 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:42.901291 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b15338-a972-43c1-b21b-d8bb680f4580 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:42.902629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b15338-a972-43c1-b21b-d8bb680f4580 None admin] RBAC: Adding query filter params (interface=public, service_id=41578547b6b44b69a971868b5fc1ebdf, region_id=RegionOne) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:42.903373 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b15338-a972-43c1-b21b-d8bb680f4580 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:42.905628 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b15338-a972-43c1-b21b-d8bb680f4580 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:42.919886 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 434/901] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Aug 29 20:20:42 2022] GET /identity/v3/endpoints?service_id=41578547b6b44b69a971868b5fc1ebdf&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:42.921740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b15338-a972-43c1-b21b-d8bb680f4580 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:42.952267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:42.999401 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062001f-c897-4ec2-93ff-125e254b3470 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:43.035691 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfd003-764a-447a-bd98-cc7a577dac31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:43.036511 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfd003-764a-447a-bd98-cc7a577dac31 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:43.037145 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfd003-764a-447a-bd98-cc7a577dac31 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:43.037145 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfd003-764a-447a-bd98-cc7a577dac31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:43.050086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfd003-764a-447a-bd98-cc7a577dac31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:43.051451 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfd003-764a-447a-bd98-cc7a577dac31 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:43.064672 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b15338-a972-43c1-b21b-d8bb680f4580 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:43.067777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-6fcfd003-764a-447a-bd98-cc7a577dac31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:43.067777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:43.067777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:43.067777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:43.067777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:43.067777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:43.067777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:43.067777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:43.067777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:43.067777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:43.067777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:43.067777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:43.067777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:43.067777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:43.067777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:43.067777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:43.069748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:43.069748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:43.069748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:43.069748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:43.069748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:43.069748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:43.069748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:43.069748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:43.069748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:43.069748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:43.069748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:43.069748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:43.069748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:43.069748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:43.069748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:43.069748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:43.069748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:43.070656 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:43.070656 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:43.070656 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:43.072378 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:43.073922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 468/902] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:43.074540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312586a-2c33-47af-9965-b12a41068f26 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:43.074923 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312586a-2c33-47af-9965-b12a41068f26 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:43.075132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312586a-2c33-47af-9965-b12a41068f26 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:43.080905 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 469/903] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:43.080905 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312586a-2c33-47af-9965-b12a41068f26 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:43.108315 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ee5ab-1e82-49fd-b636-4115ccf71843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:43.108842 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ee5ab-1e82-49fd-b636-4115ccf71843 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:43.109110 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ee5ab-1e82-49fd-b636-4115ccf71843 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:43.109368 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ee5ab-1e82-49fd-b636-4115ccf71843 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:43.109964 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ee5ab-1e82-49fd-b636-4115ccf71843 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:43.111317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ee5ab-1e82-49fd-b636-4115ccf71843 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:43.117399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 435/904] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Aug 29 20:20:42 2022] GET /identity/v3/services => generated 1497 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:20:43.125534 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb01d8-522d-40ec-be88-fe96139455ec None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:43.126651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb01d8-522d-40ec-be88-fe96139455ec None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:43.127083 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb01d8-522d-40ec-be88-fe96139455ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:43.156296 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-cddb01d8-522d-40ec-be88-fe96139455ec None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:43.158671 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-cddb01d8-522d-40ec-be88-fe96139455ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:43.202893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312586a-2c33-47af-9965-b12a41068f26 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:43.218917 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:43.245070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9808e-4a98-4651-a30b-91b7216a75a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:43.246399 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9808e-4a98-4651-a30b-91b7216a75a9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:43.246856 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9808e-4a98-4651-a30b-91b7216a75a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:43.247227 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9808e-4a98-4651-a30b-91b7216a75a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:43.248050 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9808e-4a98-4651-a30b-91b7216a75a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:43.248683 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9808e-4a98-4651-a30b-91b7216a75a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:43.250043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9808e-4a98-4651-a30b-91b7216a75a9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:43.268849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 470/905] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:43.269409 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 436/906] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:43.270529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9808e-4a98-4651-a30b-91b7216a75a9 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:43.399562 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9808e-4a98-4651-a30b-91b7216a75a9 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:43.437746 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16040cc8-baf8-4c1a-a129-4ad919eba1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:43.438388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16040cc8-baf8-4c1a-a129-4ad919eba1b3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:43.438663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16040cc8-baf8-4c1a-a129-4ad919eba1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:43.438935 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16040cc8-baf8-4c1a-a129-4ad919eba1b3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:43.446560 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16040cc8-baf8-4c1a-a129-4ad919eba1b3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:43.448087 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16040cc8-baf8-4c1a-a129-4ad919eba1b3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:43.460824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-16040cc8-baf8-4c1a-a129-4ad919eba1b3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:20:43.460824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:43.460824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:43.460824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:43.460824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:43.460824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:43.460824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:43.460824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:43.460824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:43.460824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:43.460824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:43.460824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:43.460824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:43.460824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:43.460824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:43.460824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:43.461786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:43.461786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:43.461786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:43.461786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:43.461786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:43.461786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:43.461786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:43.461786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:43.461786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:43.461786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:43.461786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:43.461786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:43.461786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:43.461786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:43.461786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:43.461786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:43.461786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:43.461786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:43.463153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:43.463153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:43.463153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:43.463153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:43.463153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:43.463153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:43.463153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:43.463153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:43.463153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:20:43.463153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:43.463153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 471/907] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Aug 29 20:20:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:43.468485 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb01d8-522d-40ec-be88-fe96139455ec None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:43.578727 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16040cc8-baf8-4c1a-a129-4ad919eba1b3 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:43.580727 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:43.584973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb01d8-522d-40ec-be88-fe96139455ec None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:43.625781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:43.626228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b3edd-c6d7-4493-914e-1017d08da683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:43.627125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b3edd-c6d7-4493-914e-1017d08da683 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:43.627666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b3edd-c6d7-4493-914e-1017d08da683 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:43.628017 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b3edd-c6d7-4493-914e-1017d08da683 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:43.628734 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562b3edd-c6d7-4493-914e-1017d08da683 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:43.629046 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562b3edd-c6d7-4493-914e-1017d08da683 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:43.630483 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562b3edd-c6d7-4493-914e-1017d08da683 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:43.655789 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 437/908] 10.222.0.18 () {56 vars in 1149 bytes} [Mon Aug 29 20:20:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:43.657295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b3edd-c6d7-4493-914e-1017d08da683 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:43.717180 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16040cc8-baf8-4c1a-a129-4ad919eba1b3 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:43.774019 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426812f-af5c-45fa-b1bf-b2135600255d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:43.774828 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426812f-af5c-45fa-b1bf-b2135600255d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:43.775121 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426812f-af5c-45fa-b1bf-b2135600255d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:43.775403 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426812f-af5c-45fa-b1bf-b2135600255d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:43.776046 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7426812f-af5c-45fa-b1bf-b2135600255d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:43.778300 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7426812f-af5c-45fa-b1bf-b2135600255d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:43.789689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-7426812f-af5c-45fa-b1bf-b2135600255d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:20:43.789689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:43.789689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:43.789689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:43.789689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:43.789689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:43.789689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:43.789689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:43.789689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:20:43.789689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:43.789689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:43.789689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:43.789689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:43.789689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:43.789689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:43.789689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:43.789689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:43.799514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:43.799514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:43.799514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:43.799514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:43.799514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:43.799514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:43.799514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:43.799514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:43.799514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:43.799514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:43.799514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:43.799514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:43.799514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:43.799514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:43.799514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:43.799514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:43.801705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:43.801705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:43.801705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:43.801705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:43.801705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:43.801705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:43.801705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:43.801705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:43.801705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:43.801705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:43.801705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:43.801705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:43.801705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 29 20:20:43.801705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:43.801705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 472/909] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:20:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:43.803751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426812f-af5c-45fa-b1bf-b2135600255d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:43.811993 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b3edd-c6d7-4493-914e-1017d08da683 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:43.867084 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b241b9a-7e2e-4ef1-8fea-de28c489fbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:43.867953 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b241b9a-7e2e-4ef1-8fea-de28c489fbeb None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:43.868343 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b241b9a-7e2e-4ef1-8fea-de28c489fbeb None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:43.868708 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b241b9a-7e2e-4ef1-8fea-de28c489fbeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:43.906733 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:43.950262 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:43.957261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426812f-af5c-45fa-b1bf-b2135600255d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:43.958381 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b241b9a-7e2e-4ef1-8fea-de28c489fbeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:43.960090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b241b9a-7e2e-4ef1-8fea-de28c489fbeb None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:44.011073 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146468e-57ca-41d9-abb0-0aa49018af47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:44.012530 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146468e-57ca-41d9-abb0-0aa49018af47 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:44.013273 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146468e-57ca-41d9-abb0-0aa49018af47 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:44.013792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146468e-57ca-41d9-abb0-0aa49018af47 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:44.014596 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146468e-57ca-41d9-abb0-0aa49018af47 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:44.015111 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146468e-57ca-41d9-abb0-0aa49018af47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:44.016740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146468e-57ca-41d9-abb0-0aa49018af47 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:44.023911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 473/910] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:20:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:44.030075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146468e-57ca-41d9-abb0-0aa49018af47 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:44.086015 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 438/911] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:20:43 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:20:44.091353 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b241b9a-7e2e-4ef1-8fea-de28c489fbeb None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:44.161822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:44.170293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146468e-57ca-41d9-abb0-0aa49018af47 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:44.205952 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9e405-86a1-449f-9653-aeda72842839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:44.206522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9e405-86a1-449f-9653-aeda72842839 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:44.206802 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9e405-86a1-449f-9653-aeda72842839 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:44.207073 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9e405-86a1-449f-9653-aeda72842839 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:44.208041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9e405-86a1-449f-9653-aeda72842839 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:44.208149 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9e405-86a1-449f-9653-aeda72842839 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:44.209456 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9e405-86a1-449f-9653-aeda72842839 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:44.218331 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b241b9a-7e2e-4ef1-8fea-de28c489fbeb None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:44.219880 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 474/912] 10.222.0.18 () {56 vars in 1114 bytes} [Mon Aug 29 20:20:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:44.220791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9e405-86a1-449f-9653-aeda72842839 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:44.247654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:44.266938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c8995-4e15-4662-8db1-ed88912f50d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:44.267752 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c8995-4e15-4662-8db1-ed88912f50d6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:44.268327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c8995-4e15-4662-8db1-ed88912f50d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:44.268610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c8995-4e15-4662-8db1-ed88912f50d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:44.282802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c8995-4e15-4662-8db1-ed88912f50d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:44.287075 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c8995-4e15-4662-8db1-ed88912f50d6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:44.312657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-8e4c8995-4e15-4662-8db1-ed88912f50d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:44.312657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:44.312657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:44.312657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:44.312657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:44.312657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:44.312657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:44.312657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:44.312657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:44.312657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:44.312657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:44.312657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:44.312657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:44.312657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:44.312657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:44.312657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:44.313590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:44.313590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:44.313590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:44.313590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:44.313590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:44.313590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:44.313590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:44.313590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:44.313590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:44.313590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:44.313590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:44.313590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:44.313590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:44.313590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:44.313590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:44.313590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:44.313590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:44.314455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:44.314455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:44.314455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:44.318978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 439/913] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:44.319909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c8995-4e15-4662-8db1-ed88912f50d6 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:44.356249 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9e405-86a1-449f-9653-aeda72842839 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:44.399524 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8a407-26a0-47a0-a141-20c653ba5534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:44.400167 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8a407-26a0-47a0-a141-20c653ba5534 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:44.400452 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8a407-26a0-47a0-a141-20c653ba5534 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:44.400722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8a407-26a0-47a0-a141-20c653ba5534 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:44.401399 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8a407-26a0-47a0-a141-20c653ba5534 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:44.402882 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8a407-26a0-47a0-a141-20c653ba5534 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:44.425247 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 475/914] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:20:44 2022] POST /identity/v3/endpoints => generated 346 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:20:44.426370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8e29d-18ad-4fdb-93e6-7cf611a63dc4 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:44.426655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8e29d-18ad-4fdb-93e6-7cf611a63dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:44.426925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8e29d-18ad-4fdb-93e6-7cf611a63dc4 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:44.428331 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 476/915] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:44.434095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7803a-2af4-4285-8463-3e2d3b7a2380 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:44.434404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7803a-2af4-4285-8463-3e2d3b7a2380 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:44.434652 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7803a-2af4-4285-8463-3e2d3b7a2380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:44.450649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c8995-4e15-4662-8db1-ed88912f50d6 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:44.454184 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a8f7803a-2af4-4285-8463-3e2d3b7a2380 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:44.455531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a8f7803a-2af4-4285-8463-3e2d3b7a2380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:44.465087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:44.503640 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38c9d6-661d-4ff3-a28d-d03c4eb34fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:44.504483 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38c9d6-661d-4ff3-a28d-d03c4eb34fbd None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:44.504843 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38c9d6-661d-4ff3-a28d-d03c4eb34fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:44.505186 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38c9d6-661d-4ff3-a28d-d03c4eb34fbd None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:44.506035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a38c9d6-661d-4ff3-a28d-d03c4eb34fbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:44.506463 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a38c9d6-661d-4ff3-a28d-d03c4eb34fbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:44.507983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a38c9d6-661d-4ff3-a28d-d03c4eb34fbd None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:44.523684 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 440/916] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:44.530254 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38c9d6-661d-4ff3-a28d-d03c4eb34fbd None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:44.533728 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 477/917] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:20:44.639121 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38c9d6-661d-4ff3-a28d-d03c4eb34fbd None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:44.675061 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc03a10-b7b8-4a92-924e-e184bbb67e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:44.675753 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc03a10-b7b8-4a92-924e-e184bbb67e2a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:44.676006 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc03a10-b7b8-4a92-924e-e184bbb67e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:44.676284 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc03a10-b7b8-4a92-924e-e184bbb67e2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:44.684400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc03a10-b7b8-4a92-924e-e184bbb67e2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:44.685850 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc03a10-b7b8-4a92-924e-e184bbb67e2a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:44.697795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-2bc03a10-b7b8-4a92-924e-e184bbb67e2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:44.697795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:44.697795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:44.697795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:44.697795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:44.697795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:44.697795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:44.697795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:44.697795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:44.697795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:44.697795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:44.697795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:44.697795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:44.697795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:44.697795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:44.699253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:44.699253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:44.699253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:44.699253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:44.699253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:44.699253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:44.699253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:44.699253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:44.699253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:44.699253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:44.699253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:44.699253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:44.699253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:44.699253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:44.699253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:44.699253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:44.699253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:44.700154 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:44.700154 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:44.700154 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:44.700154 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:44.700154 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:44.700154 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:44.700154 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:44.700154 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:44.700154 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 441/918] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:44.708894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7803a-2af4-4285-8463-3e2d3b7a2380 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:44.722497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:44.812533 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc03a10-b7b8-4a92-924e-e184bbb67e2a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:44.819353 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7803a-2af4-4285-8463-3e2d3b7a2380 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:44.847038 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:44.857992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e6398-7b12-42a3-aa8d-0e6c58a3940b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:44.858730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e6398-7b12-42a3-aa8d-0e6c58a3940b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:44.859020 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e6398-7b12-42a3-aa8d-0e6c58a3940b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:44.859299 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e6398-7b12-42a3-aa8d-0e6c58a3940b None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:44.860040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e6398-7b12-42a3-aa8d-0e6c58a3940b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:44.860352 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e6398-7b12-42a3-aa8d-0e6c58a3940b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:44.861950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e6398-7b12-42a3-aa8d-0e6c58a3940b None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:44.877219 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 478/919] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:44.882921 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e6398-7b12-42a3-aa8d-0e6c58a3940b None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:44.934229 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc03a10-b7b8-4a92-924e-e184bbb67e2a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:44.978674 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c237c6-c0a7-42a2-bac8-8801f4749dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:44.979533 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c237c6-c0a7-42a2-bac8-8801f4749dab None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:44.980350 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c237c6-c0a7-42a2-bac8-8801f4749dab None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:44.980719 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c237c6-c0a7-42a2-bac8-8801f4749dab None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:44.981665 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c237c6-c0a7-42a2-bac8-8801f4749dab None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:44.983128 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c237c6-c0a7-42a2-bac8-8801f4749dab None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:44.988496 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:44.997559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-58c237c6-c0a7-42a2-bac8-8801f4749dab None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:20:44.997559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:44.997559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:44.997559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:44.997559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:44.997559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:44.997559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:44.997559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:44.997559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:20:44.997559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:44.997559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:44.997559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:44.997559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:44.997559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:44.997559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:44.997559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:44.997559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:44.998747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:44.998747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:44.998747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:44.998747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:44.998747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:44.998747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:44.998747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:44.998747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:44.998747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:44.998747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:44.998747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:44.998747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:44.998747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:44.998747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:44.998747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:44.998747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:44.999579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:44.999579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:44.999579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:44.999579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:44.999579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:44.999579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:44.999579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:44.999579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:44.999579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:44.999579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:44.999579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:44.999579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:44.999579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:20:44.999579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:45.004020 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 442/920] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 29 20:20:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:45.006379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c237c6-c0a7-42a2-bac8-8801f4749dab None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:45.031816 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e6398-7b12-42a3-aa8d-0e6c58a3940b None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:45.081874 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ecd4d-64a0-4092-8215-fa957892f019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:45.083467 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ecd4d-64a0-4092-8215-fa957892f019 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:45.084244 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ecd4d-64a0-4092-8215-fa957892f019 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:45.084744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ecd4d-64a0-4092-8215-fa957892f019 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:45.085694 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ecd4d-64a0-4092-8215-fa957892f019 None admin] RBAC: Adding query filter params (role.id=8ece00b091b347179365b72919fdd7dc, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=61709ccc1e244185b252179451cd860f) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:45.086199 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ecd4d-64a0-4092-8215-fa957892f019 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:45.088679 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ecd4d-64a0-4092-8215-fa957892f019 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:45.096979 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 479/921] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:20:44 2022] GET /identity/v3/role_assignments?user.id=61709ccc1e244185b252179451cd860f&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=8ece00b091b347179365b72919fdd7dc => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:45.121953 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:45.133710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c237c6-c0a7-42a2-bac8-8801f4749dab None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:45.175584 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44015cc-8297-4fc1-9365-6d80033c0363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:45.176273 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44015cc-8297-4fc1-9365-6d80033c0363 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:45.176552 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44015cc-8297-4fc1-9365-6d80033c0363 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:45.176824 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44015cc-8297-4fc1-9365-6d80033c0363 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:45.177470 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44015cc-8297-4fc1-9365-6d80033c0363 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:45.177770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44015cc-8297-4fc1-9365-6d80033c0363 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:45.179172 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44015cc-8297-4fc1-9365-6d80033c0363 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:45.185225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 443/922] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Aug 29 20:20:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:45.189241 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f9ee5-713a-41fc-b44f-d863cd7810e7 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:45.189581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f9ee5-713a-41fc-b44f-d863cd7810e7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:45.189864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f9ee5-713a-41fc-b44f-d863cd7810e7 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:45.191847 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 480/923] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:45.192433 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44015cc-8297-4fc1-9365-6d80033c0363 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:45.196779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cfc24-02f7-496e-98bc-3dd9eabc3f0d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:45.197106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cfc24-02f7-496e-98bc-3dd9eabc3f0d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:45.197332 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0cfc24-02f7-496e-98bc-3dd9eabc3f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:45.218292 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-0a0cfc24-02f7-496e-98bc-3dd9eabc3f0d None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:45.219557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0a0cfc24-02f7-496e-98bc-3dd9eabc3f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:45.300832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 481/924] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:20:45.320577 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44015cc-8297-4fc1-9365-6d80033c0363 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:45.357321 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c3d2a-37a5-40c8-8dd4-a14bcc38c92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:45.357925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c3d2a-37a5-40c8-8dd4-a14bcc38c92c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:45.358249 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c3d2a-37a5-40c8-8dd4-a14bcc38c92c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:45.358522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c3d2a-37a5-40c8-8dd4-a14bcc38c92c None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:45.359145 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c3d2a-37a5-40c8-8dd4-a14bcc38c92c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:45.359442 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c3d2a-37a5-40c8-8dd4-a14bcc38c92c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:45.361087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c3d2a-37a5-40c8-8dd4-a14bcc38c92c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:45.371156 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 444/925] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Aug 29 20:20:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:45.373113 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cfc24-02f7-496e-98bc-3dd9eabc3f0d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:45.437813 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:45.486618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0cfc24-02f7-496e-98bc-3dd9eabc3f0d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:45.525108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583dbcc0-1dc6-4aec-b6a5-8bf4e7f3df92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:45.525735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583dbcc0-1dc6-4aec-b6a5-8bf4e7f3df92 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:45.526012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583dbcc0-1dc6-4aec-b6a5-8bf4e7f3df92 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:45.526281 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583dbcc0-1dc6-4aec-b6a5-8bf4e7f3df92 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:45.527087 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583dbcc0-1dc6-4aec-b6a5-8bf4e7f3df92 None admin] RBAC: Adding query filter params (interface=public, service_id=f565229009fe43448bd3ccede1fc5cd3, region_id=RegionOne) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:45.527392 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583dbcc0-1dc6-4aec-b6a5-8bf4e7f3df92 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:45.528812 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583dbcc0-1dc6-4aec-b6a5-8bf4e7f3df92 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:45.540054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 482/926] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Aug 29 20:20:45 2022] GET /identity/v3/endpoints?service_id=f565229009fe43448bd3ccede1fc5cd3&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:45.542104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c3d2a-37a5-40c8-8dd4-a14bcc38c92c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:45.570286 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:45.578719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583dbcc0-1dc6-4aec-b6a5-8bf4e7f3df92 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:45.672515 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c3d2a-37a5-40c8-8dd4-a14bcc38c92c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:45.709564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4996fb-5580-4486-bff6-411b7edd1f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:45.709770 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583dbcc0-1dc6-4aec-b6a5-8bf4e7f3df92 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:45.710090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4996fb-5580-4486-bff6-411b7edd1f90 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:45.710363 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4996fb-5580-4486-bff6-411b7edd1f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:45.710631 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4996fb-5580-4486-bff6-411b7edd1f90 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:45.711348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4996fb-5580-4486-bff6-411b7edd1f90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:45.712866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4996fb-5580-4486-bff6-411b7edd1f90 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:45.721545 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 445/927] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Aug 29 20:20:45 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:20:45.748424 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:45.751334 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff5b9b-9f27-41bb-a1bd-93f0d04811a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:45.752225 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff5b9b-9f27-41bb-a1bd-93f0d04811a3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:45.752474 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff5b9b-9f27-41bb-a1bd-93f0d04811a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:45.752753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff5b9b-9f27-41bb-a1bd-93f0d04811a3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:45.753350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff5b9b-9f27-41bb-a1bd-93f0d04811a3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:45.754739 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff5b9b-9f27-41bb-a1bd-93f0d04811a3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:45.764834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-e2ff5b9b-9f27-41bb-a1bd-93f0d04811a3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:20:45.764834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:45.764834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:45.764834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:45.764834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:45.764834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:45.764834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:45.764834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:45.764834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:20:45.764834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:45.764834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:45.764834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:45.764834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:45.764834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:45.764834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:45.764834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:45.765849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:45.765849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:45.765849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:45.765849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:45.765849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:45.765849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:45.765849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:45.765849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:45.765849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:45.765849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:45.765849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:45.765849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:45.765849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:45.765849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:45.765849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:45.765849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:45.765849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:45.766763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:45.766763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:45.766763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:45.766763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:45.766763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:45.766763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:45.766763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:45.766763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:45.766763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:45.766763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:45.766763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:45.766763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:45.766763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 29 20:20:45.766763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:45.772122 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4996fb-5580-4486-bff6-411b7edd1f90 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:45.772576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 483/928] 10.222.0.18 () {56 vars in 1132 bytes} [Mon Aug 29 20:20:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:45.864792 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:45.885632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4996fb-5580-4486-bff6-411b7edd1f90 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:45.921405 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86bd83-f38e-4fee-9ddc-76b3eb46620c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:45.922012 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86bd83-f38e-4fee-9ddc-76b3eb46620c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:45.922286 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86bd83-f38e-4fee-9ddc-76b3eb46620c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:45.922553 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86bd83-f38e-4fee-9ddc-76b3eb46620c None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:45.923175 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae86bd83-f38e-4fee-9ddc-76b3eb46620c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:45.923473 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae86bd83-f38e-4fee-9ddc-76b3eb46620c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:45.924858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae86bd83-f38e-4fee-9ddc-76b3eb46620c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:45.929971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 446/929] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 29 20:20:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:45.936225 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff5b9b-9f27-41bb-a1bd-93f0d04811a3 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:46.039526 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff5b9b-9f27-41bb-a1bd-93f0d04811a3 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:46.074281 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13734a19-5b1a-4bb4-838c-b1a38e937a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:46.074964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13734a19-5b1a-4bb4-838c-b1a38e937a7d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:46.075245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13734a19-5b1a-4bb4-838c-b1a38e937a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:46.075514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13734a19-5b1a-4bb4-838c-b1a38e937a7d None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:46.076174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13734a19-5b1a-4bb4-838c-b1a38e937a7d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:46.076476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13734a19-5b1a-4bb4-838c-b1a38e937a7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:46.077836 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13734a19-5b1a-4bb4-838c-b1a38e937a7d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:46.083886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 484/930] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 29 20:20:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:46.089436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86bd83-f38e-4fee-9ddc-76b3eb46620c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:46.194956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86bd83-f38e-4fee-9ddc-76b3eb46620c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:46.230956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc274d-0ca0-46cd-8d56-30fb70655463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:46.231559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc274d-0ca0-46cd-8d56-30fb70655463 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:46.231832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc274d-0ca0-46cd-8d56-30fb70655463 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:46.232126 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc274d-0ca0-46cd-8d56-30fb70655463 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:46.232785 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc274d-0ca0-46cd-8d56-30fb70655463 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:46.234134 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc274d-0ca0-46cd-8d56-30fb70655463 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:46.260635 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 447/931] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:20:46 2022] POST /identity/v3/endpoints => generated 347 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:20:46.544701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb856e98-4b35-412e-96ce-ba40c5a11242 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:46.545097 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb856e98-4b35-412e-96ce-ba40c5a11242 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:46.545274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb856e98-4b35-412e-96ce-ba40c5a11242 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:46.546827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 485/932] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:46.552150 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ed91a-b00b-4719-91cd-8dd3b70b7695 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:46.552439 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ed91a-b00b-4719-91cd-8dd3b70b7695 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:46.552717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ed91a-b00b-4719-91cd-8dd3b70b7695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:46.559967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:46.575241 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c58ed91a-b00b-4719-91cd-8dd3b70b7695 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:46.576462 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c58ed91a-b00b-4719-91cd-8dd3b70b7695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:46.658170 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 448/933] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:20:46.928154 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb856e98-4b35-412e-96ce-ba40c5a11242 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:46.989610 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:47.036223 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb856e98-4b35-412e-96ce-ba40c5a11242 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:47.071416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a8089-d1bd-4873-bfa1-0c05fbb798a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:47.072171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a8089-d1bd-4873-bfa1-0c05fbb798a6 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:47.072482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a8089-d1bd-4873-bfa1-0c05fbb798a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:47.072761 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a8089-d1bd-4873-bfa1-0c05fbb798a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:47.073453 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a8089-d1bd-4873-bfa1-0c05fbb798a6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:47.074877 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a8089-d1bd-4873-bfa1-0c05fbb798a6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:47.077492 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: WARNING keystone.server.flask.common [None req-504a8089-d1bd-4873-bfa1-0c05fbb798a6 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 29 20:20:47.097789 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.sql.core [None req-504a8089-d1bd-4873-bfa1-0c05fbb798a6 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 20:20:47.097789 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 20:20:47.097789 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [parameters: {'id': '3697b608ac25464da59e4052b5ef1e0d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 20:20:47.097789 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81398) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 20:20:47.114764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-504a8089-d1bd-4873-bfa1-0c05fbb798a6 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3697b608ac25464da59e4052b5ef1e0d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3697b608ac25464da59e4052b5ef1e0d. Aug 29 20:20:47.114764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:47.114764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 20:20:47.114764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 20:20:47.114764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 20:20:47.114764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 20:20:47.114764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 20:20:47.114764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 20:20:47.114764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 20:20:47.114764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application conn.query(q) Aug 29 20:20:47.114764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 20:20:47.114764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 20:20:47.114764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 20:20:47.115578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application result.read() Aug 29 20:20:47.115578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 20:20:47.115578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 20:20:47.115578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 20:20:47.115578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 20:20:47.115578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 20:20:47.115578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 20:20:47.115578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 20:20:47.115578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 20:20:47.115578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 20:20:47.115578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:47.115578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 20:20:47.115578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:47.115578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:47.115578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 20:20:47.115578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 20:20:47.115578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 29 20:20:47.115578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 29 20:20:47.116541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 20:20:47.116541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application next(self.gen) Aug 29 20:20:47.116541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 20:20:47.116541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application yield resource Aug 29 20:20:47.116541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 20:20:47.116541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application next(self.gen) Aug 29 20:20:47.116541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 20:20:47.116541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 20:20:47.116541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 20:20:47.116541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 20:20:47.116541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 20:20:47.116541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise self.value Aug 29 20:20:47.116541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 20:20:47.116541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 20:20:47.116541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 20:20:47.116541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application session.commit() Aug 29 20:20:47.116541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 20:20:47.117411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 20:20:47.117411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 20:20:47.117411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 20:20:47.117411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 20:20:47.117411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self.session.flush() Aug 29 20:20:47.117411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 20:20:47.117411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 20:20:47.117411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 20:20:47.117411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 20:20:47.117411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 20:20:47.117411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application compat.raise_( Aug 29 20:20:47.117411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 20:20:47.117411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception Aug 29 20:20:47.117411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 20:20:47.117411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 20:20:47.117411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 20:20:47.117411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 20:20:47.118370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 20:20:47.118370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 20:20:47.118370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 20:20:47.118370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 20:20:47.118370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 29 20:20:47.118370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 29 20:20:47.118370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 20:20:47.118370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 20:20:47.118370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 20:20:47.118370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 20:20:47.118370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 20:20:47.118370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 20:20:47.118370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 20:20:47.118370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 20:20:47.118370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 20:20:47.119151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 20:20:47.119151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 20:20:47.119151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception Aug 29 20:20:47.119151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 20:20:47.119151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 20:20:47.119151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 20:20:47.119151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 20:20:47.119151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 20:20:47.119151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 20:20:47.119151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 20:20:47.119151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application conn.query(q) Aug 29 20:20:47.119151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 20:20:47.119151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 20:20:47.119151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 20:20:47.119151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application result.read() Aug 29 20:20:47.119151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 20:20:47.119151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 20:20:47.121067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 20:20:47.121067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 20:20:47.121067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 20:20:47.121067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 20:20:47.121067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 20:20:47.121067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 20:20:47.121067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 29 20:20:47.121067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 29 20:20:47.121067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [parameters: {'id': '3697b608ac25464da59e4052b5ef1e0d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 29 20:20:47.121067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 20:20:47.121067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:47.121067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:47.121067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:47.121067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:47.121067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 29 20:20:47.122894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 29 20:20:47.122894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 20:20:47.122894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 20:20:47.122894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 29 20:20:47.122894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:47.122894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:47.122894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:47.122894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:47.122894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:47.122894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:47.122894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:47.122894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:47.122894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:47.122894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:47.122894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:47.122894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:47.122894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:47.124729 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:47.124729 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 29 20:20:47.124729 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 29 20:20:47.124729 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:47.124729 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:47.124729 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 29 20:20:47.124729 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 29 20:20:47.124729 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3697b608ac25464da59e4052b5ef1e0d. Aug 29 20:20:47.124729 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:47.124729 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 486/934] 10.222.0.18 () {60 vars in 1140 bytes} [Mon Aug 29 20:20:46 2022] POST /identity/v3/projects => generated 262 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:47.125371 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ed91a-b00b-4719-91cd-8dd3b70b7695 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:47.226904 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4b7c0-4ceb-4c59-a4f5-bf514ac06dd9 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:47.227207 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4b7c0-4ceb-4c59-a4f5-bf514ac06dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:47.227541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4b7c0-4ceb-4c59-a4f5-bf514ac06dd9 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:47.236331 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 487/935] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:47.237932 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f34b4-59cd-4a8a-b5d2-746f476cff66 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:47.238312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f34b4-59cd-4a8a-b5d2-746f476cff66 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:47.238594 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f34b4-59cd-4a8a-b5d2-746f476cff66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:47.242604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ed91a-b00b-4719-91cd-8dd3b70b7695 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:47.260030 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2e8f34b4-59cd-4a8a-b5d2-746f476cff66 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:47.262378 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2e8f34b4-59cd-4a8a-b5d2-746f476cff66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:47.272264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:47.281047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffed1f-e846-4672-908f-b7d0729bef4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:47.281695 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffed1f-e846-4672-908f-b7d0729bef4e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:47.281975 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffed1f-e846-4672-908f-b7d0729bef4e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:47.282247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ffed1f-e846-4672-908f-b7d0729bef4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:47.290030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ffed1f-e846-4672-908f-b7d0729bef4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:47.291690 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ffed1f-e846-4672-908f-b7d0729bef4e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:47.303691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-c5ffed1f-e846-4672-908f-b7d0729bef4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:47.303691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:47.303691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:47.303691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:47.303691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:47.303691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:47.303691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:47.303691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:47.303691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:47.303691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:47.303691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:47.303691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:47.303691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:47.303691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:47.303691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:47.305794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:47.305794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:47.305794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:47.305794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:47.305794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:47.305794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:47.305794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:47.305794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:47.305794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:47.305794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:47.305794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:47.305794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:47.305794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:47.305794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:47.305794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:47.305794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:47.305794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:47.309056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:47.309056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:47.309056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:47.309056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:47.309056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:47.309056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:47.309056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:47.309056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:47.309056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 449/936] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:47.315960 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffed1f-e846-4672-908f-b7d0729bef4e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:47.356081 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 488/937] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:20:47.422290 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ffed1f-e846-4672-908f-b7d0729bef4e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:47.458298 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39a3af-b869-4d35-ae63-0544d8001dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:47.458895 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39a3af-b869-4d35-ae63-0544d8001dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:47.459174 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39a3af-b869-4d35-ae63-0544d8001dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:47.459451 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39a3af-b869-4d35-ae63-0544d8001dc9 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:47.460147 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39a3af-b869-4d35-ae63-0544d8001dc9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:47.460443 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39a3af-b869-4d35-ae63-0544d8001dc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:47.461931 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39a3af-b869-4d35-ae63-0544d8001dc9 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:47.484254 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 450/938] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 29 20:20:47 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:47.627146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f34b4-59cd-4a8a-b5d2-746f476cff66 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:47.676036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:47.742224 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f34b4-59cd-4a8a-b5d2-746f476cff66 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:47.779968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50658fb6-8f32-49ed-ad70-0633f80a8160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:47.780612 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50658fb6-8f32-49ed-ad70-0633f80a8160 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:47.780891 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50658fb6-8f32-49ed-ad70-0633f80a8160 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:47.781164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50658fb6-8f32-49ed-ad70-0633f80a8160 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:47.792433 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50658fb6-8f32-49ed-ad70-0633f80a8160 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:47.797287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-50658fb6-8f32-49ed-ad70-0633f80a8160 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:20:47.797287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:47.797287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:47.797287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:47.797287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:47.797287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:47.797287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:47.797287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:47.797287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:47.797287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:47.797287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:47.797287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:47.797287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:47.797287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:47.797287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:47.797287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:47.799398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:47.799398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:47.799398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:47.799398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:47.799398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:47.799398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:47.799398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:47.799398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:47.799398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:47.799398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:47.799398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:47.799398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:47.799398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:47.799398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:47.799398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:47.799398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:47.801427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:47.801427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:47.801427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:47.801427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:47.801427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:47.801427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:47.801427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:47.801427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:47.801427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:47.801427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:47.801427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:20:47.801427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:47.801427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 489/939] 10.222.0.18 () {56 vars in 1120 bytes} [Mon Aug 29 20:20:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:47.805225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39a3af-b869-4d35-ae63-0544d8001dc9 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:47.818596 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47172b-b7c5-4e35-99b0-a0dadde8dd1f None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:47.818874 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47172b-b7c5-4e35-99b0-a0dadde8dd1f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:47.819148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47172b-b7c5-4e35-99b0-a0dadde8dd1f None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:47.822322 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 490/940] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:20:47.827025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db8805-8e68-464f-8acb-0112ffeb03be None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:47.827309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db8805-8e68-464f-8acb-0112ffeb03be None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:47.827584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db8805-8e68-464f-8acb-0112ffeb03be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:47.846764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b2db8805-8e68-464f-8acb-0112ffeb03be None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:47.847962 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b2db8805-8e68-464f-8acb-0112ffeb03be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:47.864251 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:47.921395 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 491/941] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:20:47.927323 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39a3af-b869-4d35-ae63-0544d8001dc9 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:47.966066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388211b-3b35-4ffb-b78e-9f0eb44bb6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:47.967445 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388211b-3b35-4ffb-b78e-9f0eb44bb6cb None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:47.968122 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388211b-3b35-4ffb-b78e-9f0eb44bb6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:47.968568 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388211b-3b35-4ffb-b78e-9f0eb44bb6cb None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:47.969398 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388211b-3b35-4ffb-b78e-9f0eb44bb6cb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:47.969922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388211b-3b35-4ffb-b78e-9f0eb44bb6cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:47.971492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388211b-3b35-4ffb-b78e-9f0eb44bb6cb None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:47.981601 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 451/942] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Aug 29 20:20:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:47.987472 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db8805-8e68-464f-8acb-0112ffeb03be None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:48.022310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:48.108238 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db8805-8e68-464f-8acb-0112ffeb03be None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:48.145093 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faaee77-b726-4292-bda1-4cb1e4e67c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:48.145812 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faaee77-b726-4292-bda1-4cb1e4e67c81 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:48.146091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faaee77-b726-4292-bda1-4cb1e4e67c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:48.146360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faaee77-b726-4292-bda1-4cb1e4e67c81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:48.159018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faaee77-b726-4292-bda1-4cb1e4e67c81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:48.160644 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faaee77-b726-4292-bda1-4cb1e4e67c81 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:48.182106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-6faaee77-b726-4292-bda1-4cb1e4e67c81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:48.182106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:48.182106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:48.182106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:48.182106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:48.182106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:48.182106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:48.182106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:48.182106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:48.182106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:48.182106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:48.182106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:48.182106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:48.182106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:48.182106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:48.182106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:48.183168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:48.183168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:48.183168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:48.183168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:48.183168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:48.183168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:48.183168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:48.183168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:48.183168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:48.183168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:48.183168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:48.183168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:48.183168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:48.183168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:48.183168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:48.183168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:48.183168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:48.184549 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:48.184549 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:48.184549 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:48.184549 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 492/943] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:48.189781 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388211b-3b35-4ffb-b78e-9f0eb44bb6cb None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:48.207040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faaee77-b726-4292-bda1-4cb1e4e67c81 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:48.302351 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388211b-3b35-4ffb-b78e-9f0eb44bb6cb None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:48.311077 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:48.315250 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:48.330041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faaee77-b726-4292-bda1-4cb1e4e67c81 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:48.347696 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20950a00-ab6c-4af1-837a-f8c8ad582148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:48.348458 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20950a00-ab6c-4af1-837a-f8c8ad582148 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:48.348736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20950a00-ab6c-4af1-837a-f8c8ad582148 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:48.349006 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20950a00-ab6c-4af1-837a-f8c8ad582148 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:48.349657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20950a00-ab6c-4af1-837a-f8c8ad582148 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:48.349957 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20950a00-ab6c-4af1-837a-f8c8ad582148 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:48.351315 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20950a00-ab6c-4af1-837a-f8c8ad582148 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:48.363475 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 452/944] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:48.364688 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c792f0e-c117-42c8-850a-97e169ae4f64 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:48.364962 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c792f0e-c117-42c8-850a-97e169ae4f64 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:48.365229 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c792f0e-c117-42c8-850a-97e169ae4f64 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:48.366825 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 453/945] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:48.371371 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c792f0e-c117-42c8-850a-97e169ae4f64 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:48.394054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf11861-0bb0-48a4-ac9a-10f036d75b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:48.394910 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf11861-0bb0-48a4-ac9a-10f036d75b12 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:48.395562 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf11861-0bb0-48a4-ac9a-10f036d75b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:48.395562 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf11861-0bb0-48a4-ac9a-10f036d75b12 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:48.396438 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf11861-0bb0-48a4-ac9a-10f036d75b12 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:48.397395 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf11861-0bb0-48a4-ac9a-10f036d75b12 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:48.407790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-2bf11861-0bb0-48a4-ac9a-10f036d75b12 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:20:48.407790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:48.407790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:48.407790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:48.407790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:48.407790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:48.407790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:48.407790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:48.407790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:20:48.407790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:48.407790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:48.407790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:48.407790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:48.407790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:48.407790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:48.407790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:48.407790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:48.408964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:48.408964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:48.408964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:48.408964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:48.408964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:48.408964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:48.408964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:48.408964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:48.408964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:48.408964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:48.408964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:48.408964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:48.408964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:48.408964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:48.408964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:48.408964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:48.409841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:48.409841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:48.409841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:48.409841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:48.409841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:48.409841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:48.409841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:48.409841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:48.409841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:48.409841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:48.409841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:48.409841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:48.409841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:20:48.409841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:48.409841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 493/946] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 29 20:20:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:48.411185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdb70d-8880-424f-aebb-69fc3fa6f22c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:48.411185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdb70d-8880-424f-aebb-69fc3fa6f22c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:48.411185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fdb70d-8880-424f-aebb-69fc3fa6f22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:48.429705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-02fdb70d-8880-424f-aebb-69fc3fa6f22c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:48.430890 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-02fdb70d-8880-424f-aebb-69fc3fa6f22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:48.505814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c792f0e-c117-42c8-850a-97e169ae4f64 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:48.512473 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 494/947] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:48.521232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fdb70d-8880-424f-aebb-69fc3fa6f22c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:48.553643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a89d4-f189-4a9e-bcf0-c8b9931f4c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:48.554237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a89d4-f189-4a9e-bcf0-c8b9931f4c68 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:48.554522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a89d4-f189-4a9e-bcf0-c8b9931f4c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:48.554801 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a89d4-f189-4a9e-bcf0-c8b9931f4c68 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:48.566452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a89d4-f189-4a9e-bcf0-c8b9931f4c68 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:48.567920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a89d4-f189-4a9e-bcf0-c8b9931f4c68 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:48.585043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-ff9a89d4-f189-4a9e-bcf0-c8b9931f4c68 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:20:48.585043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:48.585043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:48.585043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:48.585043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:48.585043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:48.585043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:48.585043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:48.585043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:48.585043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:48.585043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:48.585043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:48.585043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:48.585043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:48.585043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:48.585043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:48.586077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:48.586077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:48.586077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:48.586077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:48.586077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:48.586077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:48.586077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:48.586077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:48.586077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:48.586077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:48.586077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:48.586077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:48.586077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:48.586077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:48.586077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:48.586077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:48.586077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:48.586077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:48.587005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:48.587005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:48.587005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:48.587005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:48.587005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:48.587005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:48.587005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:48.587005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:48.587005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:20:48.587005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:48.591222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 454/948] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Aug 29 20:20:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:48.593056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a89d4-f189-4a9e-bcf0-c8b9931f4c68 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:48.630665 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:48.642540 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:48.664965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fdb70d-8880-424f-aebb-69fc3fa6f22c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:48.711235 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4be68-99c3-462e-846d-f8bad10ec093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:48.712185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4be68-99c3-462e-846d-f8bad10ec093 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:48.712475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4be68-99c3-462e-846d-f8bad10ec093 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:48.712754 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4be68-99c3-462e-846d-f8bad10ec093 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:48.713408 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4be68-99c3-462e-846d-f8bad10ec093 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:48.713713 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4be68-99c3-462e-846d-f8bad10ec093 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:48.715327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4be68-99c3-462e-846d-f8bad10ec093 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:48.724053 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 495/949] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Aug 29 20:20:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:48.727566 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4be68-99c3-462e-846d-f8bad10ec093 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:48.741363 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a89d4-f189-4a9e-bcf0-c8b9931f4c68 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:48.789234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd5901-4943-4a2d-931c-2fe39e28f13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:48.790228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd5901-4943-4a2d-931c-2fe39e28f13f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:48.790729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd5901-4943-4a2d-931c-2fe39e28f13f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:48.791196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd5901-4943-4a2d-931c-2fe39e28f13f None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:48.792659 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd5901-4943-4a2d-931c-2fe39e28f13f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:48.793194 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd5901-4943-4a2d-931c-2fe39e28f13f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:48.795190 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd5901-4943-4a2d-931c-2fe39e28f13f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:48.818250 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 455/950] 10.222.0.18 () {56 vars in 1149 bytes} [Mon Aug 29 20:20:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:48.819411 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd5901-4943-4a2d-931c-2fe39e28f13f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:48.869434 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4be68-99c3-462e-846d-f8bad10ec093 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:48.912900 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16616910-3ae3-4d58-b311-c1376395c0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:48.914181 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16616910-3ae3-4d58-b311-c1376395c0ce None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:48.914695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16616910-3ae3-4d58-b311-c1376395c0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:48.915164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16616910-3ae3-4d58-b311-c1376395c0ce None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:48.916040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16616910-3ae3-4d58-b311-c1376395c0ce None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:48.916575 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16616910-3ae3-4d58-b311-c1376395c0ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:48.918758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16616910-3ae3-4d58-b311-c1376395c0ce None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:48.929018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 496/951] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Aug 29 20:20:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:48.930145 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:48.930284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16616910-3ae3-4d58-b311-c1376395c0ce None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:48.946562 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:48.947807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd5901-4943-4a2d-931c-2fe39e28f13f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:48.996304 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e091f11-3305-4ff8-be2c-2c6583501e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:48.996967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e091f11-3305-4ff8-be2c-2c6583501e1a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:48.997254 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e091f11-3305-4ff8-be2c-2c6583501e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:48.997533 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e091f11-3305-4ff8-be2c-2c6583501e1a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:48.998171 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e091f11-3305-4ff8-be2c-2c6583501e1a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:48.999539 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e091f11-3305-4ff8-be2c-2c6583501e1a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:49.008884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-9e091f11-3305-4ff8-be2c-2c6583501e1a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:20:49.008884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:49.008884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:49.008884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:49.008884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:49.008884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:49.008884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:49.008884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:49.008884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:20:49.008884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:49.008884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:49.008884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:49.008884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:49.008884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:49.008884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:49.008884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:49.008884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:49.010338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:49.010338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:49.010338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:49.010338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:49.010338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:49.010338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:49.010338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:49.010338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:49.010338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:49.010338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:49.010338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:49.010338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:49.010338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:49.010338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:49.010338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:49.010338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:49.011537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:49.011537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:49.011537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:49.011537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:49.011537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:49.011537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:49.011537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:49.011537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:49.011537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:49.011537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:49.011537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:49.011537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:49.011537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:20:49.011537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:49.011537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 456/952] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:49.012845 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e091f11-3305-4ff8-be2c-2c6583501e1a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:49.065168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16616910-3ae3-4d58-b311-c1376395c0ce None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:49.103097 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c606d-5678-4a2c-b9de-d19bff317ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:49.103641 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c606d-5678-4a2c-b9de-d19bff317ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:49.103922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c606d-5678-4a2c-b9de-d19bff317ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:49.104222 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c606d-5678-4a2c-b9de-d19bff317ac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:49.116355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c606d-5678-4a2c-b9de-d19bff317ac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:49.117817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c606d-5678-4a2c-b9de-d19bff317ac7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:49.133741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-ef5c606d-5678-4a2c-b9de-d19bff317ac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:49.133741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:49.133741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:49.133741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:49.133741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:49.133741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:49.133741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:49.133741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:49.133741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:49.133741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:49.133741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:49.133741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:49.133741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:49.133741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:49.133741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:49.133741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:49.135312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:49.135312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:49.135312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:49.135312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:49.135312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:49.135312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:49.135312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:49.135312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:49.135312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:49.135312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:49.135312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:49.135312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:49.135312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:49.135312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:49.135312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:49.135312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:49.135312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:49.136579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:49.136579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:49.136579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:49.136579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 497/953] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:49.136579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c606d-5678-4a2c-b9de-d19bff317ac7 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:49.153880 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e091f11-3305-4ff8-be2c-2c6583501e1a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:49.200264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52034092-1a28-4655-b073-9ecf4e223e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:49.200956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52034092-1a28-4655-b073-9ecf4e223e17 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:49.201207 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52034092-1a28-4655-b073-9ecf4e223e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:49.201477 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52034092-1a28-4655-b073-9ecf4e223e17 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:49.202152 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52034092-1a28-4655-b073-9ecf4e223e17 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:49.203749 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52034092-1a28-4655-b073-9ecf4e223e17 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:49.214172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:49.235972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 457/954] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:20:49 2022] POST /identity/v3/endpoints => generated 349 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:20:49.235972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52034092-1a28-4655-b073-9ecf4e223e17 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:49.250689 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:49.276171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c606d-5678-4a2c-b9de-d19bff317ac7 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:49.357217 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7272ac-e27a-4366-a29f-3032153c03a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:49.357767 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7272ac-e27a-4366-a29f-3032153c03a9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:49.358048 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7272ac-e27a-4366-a29f-3032153c03a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:49.358327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7272ac-e27a-4366-a29f-3032153c03a9 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:49.359020 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7272ac-e27a-4366-a29f-3032153c03a9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:49.359332 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7272ac-e27a-4366-a29f-3032153c03a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:49.360815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7272ac-e27a-4366-a29f-3032153c03a9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:49.369840 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 498/955] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 29 20:20:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:49.376698 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7272ac-e27a-4366-a29f-3032153c03a9 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:49.412011 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52034092-1a28-4655-b073-9ecf4e223e17 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:49.461638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ace72-5ea3-4ae5-af46-0c899b05f772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:49.462629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ace72-5ea3-4ae5-af46-0c899b05f772 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:49.463130 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ace72-5ea3-4ae5-af46-0c899b05f772 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:49.463705 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ace72-5ea3-4ae5-af46-0c899b05f772 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:49.464672 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ace72-5ea3-4ae5-af46-0c899b05f772 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:49.465171 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ace72-5ea3-4ae5-af46-0c899b05f772 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:49.467218 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ace72-5ea3-4ae5-af46-0c899b05f772 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:49.481628 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 458/956] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:49.492954 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ace72-5ea3-4ae5-af46-0c899b05f772 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:49.512901 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7272ac-e27a-4366-a29f-3032153c03a9 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:49.535862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:49.564394 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b12c21-7c8c-4933-bc6d-36049e62af75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:49.565168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b12c21-7c8c-4933-bc6d-36049e62af75 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:49.565461 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b12c21-7c8c-4933-bc6d-36049e62af75 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:49.565742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b12c21-7c8c-4933-bc6d-36049e62af75 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:49.566552 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b12c21-7c8c-4933-bc6d-36049e62af75 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:49.566868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b12c21-7c8c-4933-bc6d-36049e62af75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:49.568282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b12c21-7c8c-4933-bc6d-36049e62af75 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:49.574332 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 499/957] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 29 20:20:49 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:49.584293 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:49.639681 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ace72-5ea3-4ae5-af46-0c899b05f772 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:49.686348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e13123-6098-4ba3-9082-baf752ffa27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:49.687005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e13123-6098-4ba3-9082-baf752ffa27b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:49.687282 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e13123-6098-4ba3-9082-baf752ffa27b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:49.687552 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e13123-6098-4ba3-9082-baf752ffa27b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:49.695292 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e13123-6098-4ba3-9082-baf752ffa27b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:49.696800 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e13123-6098-4ba3-9082-baf752ffa27b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:49.701228 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6560fe-a441-416f-8e25-1947780bcdfd None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:49.701536 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6560fe-a441-416f-8e25-1947780bcdfd None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:49.701825 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6560fe-a441-416f-8e25-1947780bcdfd None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:49.703163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 500/958] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:20:49.709292 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5d1cd-2b6d-40b3-b2f0-ad8ded9583a8 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:49.709582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5d1cd-2b6d-40b3-b2f0-ad8ded9583a8 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:49.709739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-28e13123-6098-4ba3-9082-baf752ffa27b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:49.709739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:49.709739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:49.709739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:49.709739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:49.709739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:49.709739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:49.709739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:49.709739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:49.709739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:49.709739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:49.709739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:49.709739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:49.709739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:49.709739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:49.710655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:49.710655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:49.710655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:49.710655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:49.710655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:49.710655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:49.710655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:49.710655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:49.710655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:49.710655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:49.710655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:49.710655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:49.710655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:49.710655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:49.710655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:49.710655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:49.710655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:49.712260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:49.712260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:49.712260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:49.712260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:49.712260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:49.712260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:49.712260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:49.712260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:49.712684 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5d1cd-2b6d-40b3-b2f0-ad8ded9583a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:49.712766 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 459/959] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:49.717375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e13123-6098-4ba3-9082-baf752ffa27b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:49.733469 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a5d5d1cd-2b6d-40b3-b2f0-ad8ded9583a8 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:49.734701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a5d5d1cd-2b6d-40b3-b2f0-ad8ded9583a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:49.828021 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 501/960] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:49.852552 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e13123-6098-4ba3-9082-baf752ffa27b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:49.879815 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:49.889424 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861faef6-a1ca-4e5a-886f-eef087d3778a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:49.890376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861faef6-a1ca-4e5a-886f-eef087d3778a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:49.890853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861faef6-a1ca-4e5a-886f-eef087d3778a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:49.891335 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861faef6-a1ca-4e5a-886f-eef087d3778a None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:49.892249 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861faef6-a1ca-4e5a-886f-eef087d3778a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:49.892759 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861faef6-a1ca-4e5a-886f-eef087d3778a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:49.894375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861faef6-a1ca-4e5a-886f-eef087d3778a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:49.908391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 460/961] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:49.914789 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5d1cd-2b6d-40b3-b2f0-ad8ded9583a8 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:50.015998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5d1cd-2b6d-40b3-b2f0-ad8ded9583a8 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:50.029772 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:50.051204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7315a-cb98-4d96-ac96-0a6965f3c0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:50.051870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7315a-cb98-4d96-ac96-0a6965f3c0df None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:50.052175 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7315a-cb98-4d96-ac96-0a6965f3c0df None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:50.052445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7315a-cb98-4d96-ac96-0a6965f3c0df None admin] PATH_INFO: `/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/61709ccc1e244185b252179451cd860f/roles/8ece00b091b347179365b72919fdd7dc` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:50.054582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7315a-cb98-4d96-ac96-0a6965f3c0df None admin] RBAC: Authorizing `identity:create_grant(project_id=90b25dc95d5a4a068994bdd801d6b8ea, user_id=61709ccc1e244185b252179451cd860f, role_id=8ece00b091b347179365b72919fdd7dc)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:50.056015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7315a-cb98-4d96-ac96-0a6965f3c0df None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:50.074723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 502/962] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:20:49 2022] PUT /identity/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/61709ccc1e244185b252179451cd860f/roles/8ece00b091b347179365b72919fdd7dc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:20:50.105167 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861faef6-a1ca-4e5a-886f-eef087d3778a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:50.217617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861faef6-a1ca-4e5a-886f-eef087d3778a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:50.260607 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b8ec2-0a24-4f3c-8039-623335ca2bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:50.261250 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b8ec2-0a24-4f3c-8039-623335ca2bc3 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:50.261525 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b8ec2-0a24-4f3c-8039-623335ca2bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:50.261794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b8ec2-0a24-4f3c-8039-623335ca2bc3 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:50.262428 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b8ec2-0a24-4f3c-8039-623335ca2bc3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:50.263841 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b8ec2-0a24-4f3c-8039-623335ca2bc3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:50.266388 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: WARNING keystone.server.flask.common [None req-ed7b8ec2-0a24-4f3c-8039-623335ca2bc3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 29 20:20:50.281604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.sql.core [None req-ed7b8ec2-0a24-4f3c-8039-623335ca2bc3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 20:20:50.281604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 20:20:50.281604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [parameters: {'user_id': '6c12bf7a42be436b9a83e16c1879e720', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 20:20:50.281604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81397) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 29 20:20:50.289399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-ed7b8ec2-0a24-4f3c-8039-623335ca2bc3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 20:20:50.289399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:50.289399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 20:20:50.289399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 20:20:50.289399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 20:20:50.289399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 20:20:50.289399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 20:20:50.289399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 20:20:50.289399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 20:20:50.289399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application conn.query(q) Aug 29 20:20:50.289399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 20:20:50.289399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 20:20:50.289399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 20:20:50.289399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application result.read() Aug 29 20:20:50.289399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 20:20:50.290287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 20:20:50.290287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 20:20:50.290287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 20:20:50.290287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 20:20:50.290287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 20:20:50.290287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 20:20:50.290287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 20:20:50.290287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 20:20:50.290287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:50.290287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 29 20:20:50.290287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:50.290287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:50.290287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 29 20:20:50.290287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 29 20:20:50.290287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 29 20:20:50.290287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 29 20:20:50.290287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 20:20:50.290287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application next(self.gen) Aug 29 20:20:50.291849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 29 20:20:50.291849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application yield resource Aug 29 20:20:50.291849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 29 20:20:50.291849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application next(self.gen) Aug 29 20:20:50.291849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 29 20:20:50.291849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.session.rollback() Aug 29 20:20:50.291849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 29 20:20:50.291849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.force_reraise() Aug 29 20:20:50.291849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 29 20:20:50.291849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise self.value Aug 29 20:20:50.291849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 29 20:20:50.291849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 29 20:20:50.291849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 29 20:20:50.291849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application session.commit() Aug 29 20:20:50.291849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 29 20:20:50.291849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 29 20:20:50.291849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 29 20:20:50.293581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._prepare_impl() Aug 29 20:20:50.293581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 29 20:20:50.293581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.session.flush() Aug 29 20:20:50.293581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 29 20:20:50.293581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._flush(objects) Aug 29 20:20:50.293581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 29 20:20:50.293581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 29 20:20:50.293581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 29 20:20:50.293581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application compat.raise_( Aug 29 20:20:50.293581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 20:20:50.293581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception Aug 29 20:20:50.293581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 29 20:20:50.293581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application flush_context.execute() Aug 29 20:20:50.293581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 29 20:20:50.293581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rec.execute(self) Aug 29 20:20:50.293581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 29 20:20:50.293581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 29 20:20:50.294673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 29 20:20:50.294673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 29 20:20:50.294673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 29 20:20:50.294673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 29 20:20:50.294673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 29 20:20:50.294673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 29 20:20:50.294673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 29 20:20:50.294673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 29 20:20:50.294673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 29 20:20:50.294673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 29 20:20:50.294673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 29 20:20:50.294673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 29 20:20:50.294673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 29 20:20:50.294673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 29 20:20:50.294673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 29 20:20:50.294673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception Aug 29 20:20:50.295694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 29 20:20:50.295694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 29 20:20:50.295694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 29 20:20:50.295694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 29 20:20:50.295694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 29 20:20:50.295694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application result = self._query(query) Aug 29 20:20:50.295694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 29 20:20:50.295694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application conn.query(q) Aug 29 20:20:50.295694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 29 20:20:50.295694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 29 20:20:50.295694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 29 20:20:50.295694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application result.read() Aug 29 20:20:50.295694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 29 20:20:50.295694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 29 20:20:50.295694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 29 20:20:50.295694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 29 20:20:50.296764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 29 20:20:50.296764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 29 20:20:50.296764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 29 20:20:50.296764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 29 20:20:50.296764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 29 20:20:50.296764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 29 20:20:50.296764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [parameters: {'user_id': '6c12bf7a42be436b9a83e16c1879e720', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 29 20:20:50.296764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 29 20:20:50.296764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:50.296764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:50.296764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:50.296764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:50.296764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:50.296764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:50.296764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:50.297754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:50.297754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:50.297754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:50.297754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:50.297754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:50.297754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:50.297754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:50.297754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 29 20:20:50.297754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 29 20:20:50.297754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:50.297754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:50.297754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:50.297754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:50.297754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:50.297754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:50.297754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 29 20:20:50.297754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 29 20:20:50.298906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 29 20:20:50.298906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 29 20:20:50.298906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 29 20:20:50.298906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 29 20:20:50.298906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 29 20:20:50.298906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:50.298906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 461/963] 10.222.0.18 () {60 vars in 1131 bytes} [Mon Aug 29 20:20:50 2022] POST /identity/v3/users => generated 160 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:50.305569 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7315a-cb98-4d96-ac96-0a6965f3c0df None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:50.416230 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7315a-cb98-4d96-ac96-0a6965f3c0df None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:50.451831 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f956eb-b7f0-4e05-a20b-babeea9bb4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:50.452506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f956eb-b7f0-4e05-a20b-babeea9bb4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:50.452789 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f956eb-b7f0-4e05-a20b-babeea9bb4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:50.453063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f956eb-b7f0-4e05-a20b-babeea9bb4e7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:50.461279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f956eb-b7f0-4e05-a20b-babeea9bb4e7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:50.463152 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f956eb-b7f0-4e05-a20b-babeea9bb4e7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:50.477068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-85f956eb-b7f0-4e05-a20b-babeea9bb4e7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:20:50.477068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:50.477068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:50.477068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:50.477068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:50.477068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:50.477068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:50.477068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:50.477068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:50.477068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:50.477068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:50.477068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:50.477068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:50.477068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:50.477068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:50.477068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:50.479355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:50.479355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:50.479355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:50.479355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:50.479355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:50.479355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:50.479355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:50.479355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:50.479355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:50.479355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:50.479355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:50.479355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:50.479355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:50.479355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:50.479355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:50.479355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:50.479355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:50.479355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:50.482005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:50.482005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:50.482005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:50.482005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:50.482005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:50.482005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:50.482005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:50.482005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:50.482005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 29 20:20:50.482005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:50.482005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 503/964] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:20:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:50.492377 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b8ec2-0a24-4f3c-8039-623335ca2bc3 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:50.545055 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:50.598703 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b8ec2-0a24-4f3c-8039-623335ca2bc3 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:50.633329 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f315d-bbf4-4133-b81a-62518212155f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:50.634284 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f315d-bbf4-4133-b81a-62518212155f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:50.634770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f315d-bbf4-4133-b81a-62518212155f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:50.635345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f315d-bbf4-4133-b81a-62518212155f None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:50.636817 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f315d-bbf4-4133-b81a-62518212155f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:50.637326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f315d-bbf4-4133-b81a-62518212155f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:50.639474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f315d-bbf4-4133-b81a-62518212155f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:50.696102 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 462/965] 10.222.0.18 () {56 vars in 1101 bytes} [Mon Aug 29 20:20:50 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:51.321600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ada5f-8fa2-4cf1-9ba5-02b80d012d19 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:51.322094 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ada5f-8fa2-4cf1-9ba5-02b80d012d19 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:51.322171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ada5f-8fa2-4cf1-9ba5-02b80d012d19 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:51.323702 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 504/966] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:20:51.328750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230a821-a08f-4d3e-95c6-ade4030c34bc None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:51.329154 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230a821-a08f-4d3e-95c6-ade4030c34bc None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:51.329317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230a821-a08f-4d3e-95c6-ade4030c34bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:51.347455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1230a821-a08f-4d3e-95c6-ade4030c34bc None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:51.348983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1230a821-a08f-4d3e-95c6-ade4030c34bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:51.415188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 463/967] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:51.630096 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf25f1-8526-4ef8-9ede-848f23d77c8d None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:51.630300 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf25f1-8526-4ef8-9ede-848f23d77c8d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:51.630652 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf25f1-8526-4ef8-9ede-848f23d77c8d None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:51.632090 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 505/968] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:51.637016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183aaab-fa88-445d-b8c4-140728ca9e3f None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:51.637318 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183aaab-fa88-445d-b8c4-140728ca9e3f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:51.637642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183aaab-fa88-445d-b8c4-140728ca9e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:51.657317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-f183aaab-fa88-445d-b8c4-140728ca9e3f None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:51.658525 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f183aaab-fa88-445d-b8c4-140728ca9e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:51.682161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf25f1-8526-4ef8-9ede-848f23d77c8d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:51.689621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:51.716144 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:51.733616 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 464/969] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:51.789255 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf25f1-8526-4ef8-9ede-848f23d77c8d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:51.822934 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b15789-aa4e-4b2b-ba3d-af922f22a2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:51.823636 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b15789-aa4e-4b2b-ba3d-af922f22a2bd None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:51.823916 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b15789-aa4e-4b2b-ba3d-af922f22a2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:51.824263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b15789-aa4e-4b2b-ba3d-af922f22a2bd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:51.824790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b15789-aa4e-4b2b-ba3d-af922f22a2bd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:51.826127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b15789-aa4e-4b2b-ba3d-af922f22a2bd None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:51.834822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-f2b15789-aa4e-4b2b-ba3d-af922f22a2bd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:20:51.834822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:51.834822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:51.834822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:51.834822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:51.834822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:51.834822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:51.834822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:51.834822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:20:51.834822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:51.834822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:51.834822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:51.834822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:51.834822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:51.834822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:51.834822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:51.834822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:51.835832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:51.835832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:51.835832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:51.835832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:51.835832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:51.835832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:51.835832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:51.835832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:51.835832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:51.835832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:51.835832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:51.835832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:51.835832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:51.835832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:51.835832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:51.835832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:51.836745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:51.836745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:51.836745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:51.836745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:51.836745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:51.836745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:51.836745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:51.836745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:51.836745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:51.836745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:51.836745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:51.836745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:51.836745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:20:51.836745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:51.840275 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 506/970] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 29 20:20:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:51.842467 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183aaab-fa88-445d-b8c4-140728ca9e3f None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:51.940066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183aaab-fa88-445d-b8c4-140728ca9e3f None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:51.974126 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21013f-343f-4d64-9c13-87c1ca6348d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:51.974733 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21013f-343f-4d64-9c13-87c1ca6348d6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:51.975007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21013f-343f-4d64-9c13-87c1ca6348d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:51.975274 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21013f-343f-4d64-9c13-87c1ca6348d6 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:51.975898 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21013f-343f-4d64-9c13-87c1ca6348d6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:51.976232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21013f-343f-4d64-9c13-87c1ca6348d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:51.977644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21013f-343f-4d64-9c13-87c1ca6348d6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:51.987216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 465/971] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Aug 29 20:20:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:51.988789 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b15789-aa4e-4b2b-ba3d-af922f22a2bd None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:52.000075 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21013f-343f-4d64-9c13-87c1ca6348d6 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:52.036662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:52.089785 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:52.097065 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b15789-aa4e-4b2b-ba3d-af922f22a2bd None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:52.110055 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21013f-343f-4d64-9c13-87c1ca6348d6 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:52.134415 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc299b-7378-47a0-8f96-4a1e6b1b5771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:52.135026 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc299b-7378-47a0-8f96-4a1e6b1b5771 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:52.135302 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc299b-7378-47a0-8f96-4a1e6b1b5771 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:52.136162 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc299b-7378-47a0-8f96-4a1e6b1b5771 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:52.136885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc299b-7378-47a0-8f96-4a1e6b1b5771 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:52.137226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc299b-7378-47a0-8f96-4a1e6b1b5771 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:52.139211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc299b-7378-47a0-8f96-4a1e6b1b5771 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:52.146091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040f702-5d6a-458b-b2cd-7ed7083141f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:52.146881 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040f702-5d6a-458b-b2cd-7ed7083141f1 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:52.146992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 507/972] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Aug 29 20:20:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:52.147070 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040f702-5d6a-458b-b2cd-7ed7083141f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:52.147228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040f702-5d6a-458b-b2cd-7ed7083141f1 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:52.147863 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040f702-5d6a-458b-b2cd-7ed7083141f1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:52.148104 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078c871-17e6-48f6-a509-4d6fd3e478fa None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:52.148387 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078c871-17e6-48f6-a509-4d6fd3e478fa None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:52.148658 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078c871-17e6-48f6-a509-4d6fd3e478fa None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:52.149254 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040f702-5d6a-458b-b2cd-7ed7083141f1 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:52.149979 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 508/973] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:52.156467 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078c871-17e6-48f6-a509-4d6fd3e478fa None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:52.165515 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 466/974] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 29 20:20:51 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:20:52.166895 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef6137-e410-4cb0-90e4-bf0608de7e74 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:52.167177 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef6137-e410-4cb0-90e4-bf0608de7e74 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:52.167458 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef6137-e410-4cb0-90e4-bf0608de7e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:52.186654 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-55ef6137-e410-4cb0-90e4-bf0608de7e74 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:52.187906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-55ef6137-e410-4cb0-90e4-bf0608de7e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:52.272488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 467/975] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:52 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:52.275897 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078c871-17e6-48f6-a509-4d6fd3e478fa None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:52.321895 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d59afa-2eb2-48f0-bbef-b4ee54d6f289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:52.322502 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d59afa-2eb2-48f0-bbef-b4ee54d6f289 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:52.322771 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d59afa-2eb2-48f0-bbef-b4ee54d6f289 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:52.323663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d59afa-2eb2-48f0-bbef-b4ee54d6f289 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:52.324541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d59afa-2eb2-48f0-bbef-b4ee54d6f289 None admin] RBAC: Adding query filter params (interface=admin, service_id=f565229009fe43448bd3ccede1fc5cd3, region_id=RegionOne) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:52.324849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d59afa-2eb2-48f0-bbef-b4ee54d6f289 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:52.326207 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d59afa-2eb2-48f0-bbef-b4ee54d6f289 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:52.332640 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 509/976] 10.222.0.18 () {56 vars in 1249 bytes} [Mon Aug 29 20:20:52 2022] GET /identity/v3/endpoints?service_id=f565229009fe43448bd3ccede1fc5cd3&interface=admin®ion_id=RegionOne => generated 193 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:52.338577 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef6137-e410-4cb0-90e4-bf0608de7e74 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:52.359853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:52.465466 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef6137-e410-4cb0-90e4-bf0608de7e74 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:52.499944 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d4f4d-0673-4c0d-aa82-4ffc360262f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:52.500584 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d4f4d-0673-4c0d-aa82-4ffc360262f5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:52.500864 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d4f4d-0673-4c0d-aa82-4ffc360262f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:52.501136 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d4f4d-0673-4c0d-aa82-4ffc360262f5 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:52.501792 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d4f4d-0673-4c0d-aa82-4ffc360262f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:52.503220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d4f4d-0673-4c0d-aa82-4ffc360262f5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:52.511688 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 468/977] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Aug 29 20:20:52 2022] GET /identity/v3/services => generated 1727 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:20:52.554143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d59afa-2eb2-48f0-bbef-b4ee54d6f289 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:52.582914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:52.671006 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d59afa-2eb2-48f0-bbef-b4ee54d6f289 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:52.707839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0616e-e777-4de0-92e0-024e2aaa71f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:52.710192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0616e-e777-4de0-92e0-024e2aaa71f6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:52.710192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0616e-e777-4de0-92e0-024e2aaa71f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:52.710192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0616e-e777-4de0-92e0-024e2aaa71f6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:52.732698 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0616e-e777-4de0-92e0-024e2aaa71f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:20:52.739274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-d6d0616e-e777-4de0-92e0-024e2aaa71f6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:20:52.739274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:52.739274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:52.739274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:52.739274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:52.739274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:52.739274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:52.739274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:52.739274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:52.739274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:52.739274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:52.739274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:52.739274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:20:52.739274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:20:52.739274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:20:52.739274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:20:52.741547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:20:52.741547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:20:52.741547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:20:52.741547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:20:52.741547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:52.741547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:52.741547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:52.741547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:52.741547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:52.741547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:52.741547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:52.741547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:52.741547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:52.741547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:52.741547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:52.741547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:52.743709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:52.743709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:52.743709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:52.743709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:52.743709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:20:52.743709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:20:52.743709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:20:52.743709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:20:52.743709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:20:52.743709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:20:52.743709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:20:52.743709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:52.745897 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 510/978] 10.222.0.18 () {56 vars in 1120 bytes} [Mon Aug 29 20:20:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:52.747228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d4f4d-0673-4c0d-aa82-4ffc360262f5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:52.829732 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:52.849107 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d4f4d-0673-4c0d-aa82-4ffc360262f5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:52.885932 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198e2ee-b0dd-444a-9bbb-047d2ab498c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:52.886626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198e2ee-b0dd-444a-9bbb-047d2ab498c4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:52.886967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198e2ee-b0dd-444a-9bbb-047d2ab498c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:52.887295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198e2ee-b0dd-444a-9bbb-047d2ab498c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:52.888045 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198e2ee-b0dd-444a-9bbb-047d2ab498c4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:52.888408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198e2ee-b0dd-444a-9bbb-047d2ab498c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:52.889817 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0198e2ee-b0dd-444a-9bbb-047d2ab498c4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:52.902572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 469/979] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Aug 29 20:20:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:52.904091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0616e-e777-4de0-92e0-024e2aaa71f6 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:52.991340 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:53.008567 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0616e-e777-4de0-92e0-024e2aaa71f6 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:53.045673 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9c7be-e29d-4bed-bb87-c0c0eb7c0989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:53.046314 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9c7be-e29d-4bed-bb87-c0c0eb7c0989 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:53.046597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9c7be-e29d-4bed-bb87-c0c0eb7c0989 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:53.046873 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9c7be-e29d-4bed-bb87-c0c0eb7c0989 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:53.057487 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9c7be-e29d-4bed-bb87-c0c0eb7c0989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:53.059022 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9c7be-e29d-4bed-bb87-c0c0eb7c0989 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:53.074193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-c1d9c7be-e29d-4bed-bb87-c0c0eb7c0989 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:53.074193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:53.074193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:53.074193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:53.074193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:53.074193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:53.074193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:53.074193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:53.074193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:53.074193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:53.074193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:53.074193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:53.074193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:53.074193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:53.074193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:53.074193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:53.080037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:53.080037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:53.080037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:53.080037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:53.080037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:53.080037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:53.080037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:53.080037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:53.080037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:53.080037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:53.080037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:53.080037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:53.080037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:53.080037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:53.080037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:53.080037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:53.080037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:53.081004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:53.081004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:53.081004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:53.081004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 511/980] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:53.081239 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198e2ee-b0dd-444a-9bbb-047d2ab498c4 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:53.188783 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198e2ee-b0dd-444a-9bbb-047d2ab498c4 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:53.229025 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d150194-e4ce-4ce6-9cef-d7c713b70b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:53.230504 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d150194-e4ce-4ce6-9cef-d7c713b70b87 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:53.230504 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d150194-e4ce-4ce6-9cef-d7c713b70b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:53.230765 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d150194-e4ce-4ce6-9cef-d7c713b70b87 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:53.231368 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d150194-e4ce-4ce6-9cef-d7c713b70b87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:53.231673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d150194-e4ce-4ce6-9cef-d7c713b70b87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:53.233078 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d150194-e4ce-4ce6-9cef-d7c713b70b87 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:53.249687 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 470/981] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:53.250975 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9c7be-e29d-4bed-bb87-c0c0eb7c0989 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:53.365935 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9c7be-e29d-4bed-bb87-c0c0eb7c0989 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:53.400379 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804092c-2ee2-4f8b-90e1-3e9b86044242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:53.401014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804092c-2ee2-4f8b-90e1-3e9b86044242 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:53.401352 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804092c-2ee2-4f8b-90e1-3e9b86044242 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:53.401630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804092c-2ee2-4f8b-90e1-3e9b86044242 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:53.405300 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:53.410930 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804092c-2ee2-4f8b-90e1-3e9b86044242 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:53.412823 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804092c-2ee2-4f8b-90e1-3e9b86044242 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:53.424404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-5804092c-2ee2-4f8b-90e1-3e9b86044242 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:20:53.424404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:53.424404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:53.424404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:53.424404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:53.424404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:53.424404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:53.424404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:53.424404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:53.424404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:53.424404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:53.424404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:53.424404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:20:53.424404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:20:53.424404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:20:53.424404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:20:53.425359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:53.425359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:53.425359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:20:53.425359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:53.425359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:20:53.425359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:20:53.425359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:53.425359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:53.425359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:53.425359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:53.425359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:53.425359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:53.425359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:53.425359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:53.425359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:53.425359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:53.425359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:53.425359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:53.430959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:53.430959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:53.430959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:20:53.430959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:20:53.430959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:20:53.430959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:20:53.430959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:20:53.430959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:20:53.430959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 29 20:20:53.430959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:53.430959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 512/982] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Aug 29 20:20:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:53.431587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d150194-e4ce-4ce6-9cef-d7c713b70b87 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:53.436598 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:53.536780 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d150194-e4ce-4ce6-9cef-d7c713b70b87 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:53.571643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d4c7a-b649-4474-8346-30ccb23eef62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:53.572289 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d4c7a-b649-4474-8346-30ccb23eef62 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:53.572569 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d4c7a-b649-4474-8346-30ccb23eef62 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:53.572842 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d4c7a-b649-4474-8346-30ccb23eef62 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:53.574123 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d4c7a-b649-4474-8346-30ccb23eef62 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:53.574438 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d4c7a-b649-4474-8346-30ccb23eef62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:53.575867 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d4c7a-b649-4474-8346-30ccb23eef62 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:53.598585 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 471/983] 10.222.0.18 () {56 vars in 1149 bytes} [Mon Aug 29 20:20:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:53.600304 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804092c-2ee2-4f8b-90e1-3e9b86044242 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:53.706944 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804092c-2ee2-4f8b-90e1-3e9b86044242 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:53.740540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486931ca-051d-41d0-abb7-ee3dc4c4bc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:53.741155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486931ca-051d-41d0-abb7-ee3dc4c4bc4a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:53.741438 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486931ca-051d-41d0-abb7-ee3dc4c4bc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:53.741713 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486931ca-051d-41d0-abb7-ee3dc4c4bc4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:53.816104 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:53.826071 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486931ca-051d-41d0-abb7-ee3dc4c4bc4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:53.827535 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486931ca-051d-41d0-abb7-ee3dc4c4bc4a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:53.930217 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 513/984] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:20:53 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:20:53.931538 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d4c7a-b649-4474-8346-30ccb23eef62 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:53.993987 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:54.034365 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d4c7a-b649-4474-8346-30ccb23eef62 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:54.068952 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6e42c-8210-4e85-b04b-2678c82307bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:54.069581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6e42c-8210-4e85-b04b-2678c82307bb None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:54.069846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6e42c-8210-4e85-b04b-2678c82307bb None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:54.070113 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6e42c-8210-4e85-b04b-2678c82307bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:54.080825 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6e42c-8210-4e85-b04b-2678c82307bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:54.082272 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6e42c-8210-4e85-b04b-2678c82307bb None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:54.094023 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c64b27-7143-47dc-94dd-8c690777b358 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:54.094319 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c64b27-7143-47dc-94dd-8c690777b358 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:54.094578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c64b27-7143-47dc-94dd-8c690777b358 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:54.096082 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 514/985] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:54.097971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-a0b6e42c-8210-4e85-b04b-2678c82307bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:54.097971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:54.097971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:54.097971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:54.097971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:54.097971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:54.097971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:54.097971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:54.097971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:54.097971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:54.097971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:54.097971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:54.097971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:20:54.097971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:20:54.097971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:20:54.097971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:20:54.099242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:54.099242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:54.099242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:54.099242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:54.099242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:54.099242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:54.099242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:54.099242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:54.099242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:54.099242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:54.099242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:54.099242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:54.099242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:54.099242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:54.099242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:54.099242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:54.099242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:20:54.100148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:20:54.100148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:20:54.100148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:54.100148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 472/986] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:54.102128 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e9ac1-5d01-46b3-84e7-a2f3c058a728 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:54.102484 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e9ac1-5d01-46b3-84e7-a2f3c058a728 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:54.102832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e9ac1-5d01-46b3-84e7-a2f3c058a728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:54.105832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6e42c-8210-4e85-b04b-2678c82307bb None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:54.123067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-dc5e9ac1-5d01-46b3-84e7-a2f3c058a728 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:54.124405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-dc5e9ac1-5d01-46b3-84e7-a2f3c058a728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:54.207490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 515/987] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:54 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:54.214356 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6e42c-8210-4e85-b04b-2678c82307bb None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:54.228261 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:54.250692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5619c4-0f04-47db-b4a3-2c674f3b9578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:54.251226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5619c4-0f04-47db-b4a3-2c674f3b9578 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:54.251500 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5619c4-0f04-47db-b4a3-2c674f3b9578 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:54.251765 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5619c4-0f04-47db-b4a3-2c674f3b9578 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:54.252480 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5619c4-0f04-47db-b4a3-2c674f3b9578 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:54.252782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5619c4-0f04-47db-b4a3-2c674f3b9578 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:54.254133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5619c4-0f04-47db-b4a3-2c674f3b9578 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:54.266547 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 473/988] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:54.271872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e9ac1-5d01-46b3-84e7-a2f3c058a728 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:54.307115 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:54.379832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e9ac1-5d01-46b3-84e7-a2f3c058a728 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:54.415101 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b4fdb-63f9-4f61-b7ee-1a5f0a3c386c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:54.415976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b4fdb-63f9-4f61-b7ee-1a5f0a3c386c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:54.416400 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b4fdb-63f9-4f61-b7ee-1a5f0a3c386c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:54.417375 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b4fdb-63f9-4f61-b7ee-1a5f0a3c386c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:54.424206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b4fdb-63f9-4f61-b7ee-1a5f0a3c386c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:54.425779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b4fdb-63f9-4f61-b7ee-1a5f0a3c386c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:54.437719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-460b4fdb-63f9-4f61-b7ee-1a5f0a3c386c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:54.437719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:54.437719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:54.437719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:54.437719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:54.437719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:54.437719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:54.437719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:54.437719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:54.437719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:54.437719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:54.437719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:54.437719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:20:54.437719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:20:54.437719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:20:54.438689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:20:54.438689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:54.438689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:54.438689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:54.438689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:54.438689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:54.438689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:54.438689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:54.438689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:54.438689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:54.438689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:54.438689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:54.438689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:54.438689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:54.438689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:54.438689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:54.438689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:54.444369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:20:54.444369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:20:54.444369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:20:54.444369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:20:54.444369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:20:54.444369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:20:54.444369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:20:54.444369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:54.444369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 516/989] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:54.445115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5619c4-0f04-47db-b4a3-2c674f3b9578 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:54.445813 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063bb79-b3cf-46a0-b21b-c2112c5f9b31 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:54.446103 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063bb79-b3cf-46a0-b21b-c2112c5f9b31 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:54.446405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063bb79-b3cf-46a0-b21b-c2112c5f9b31 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:54.448733 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 517/990] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:54.454766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30715221-b77a-48fe-8d9d-d6b7e59e4638 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:54.455181 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30715221-b77a-48fe-8d9d-d6b7e59e4638 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:54.455458 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30715221-b77a-48fe-8d9d-d6b7e59e4638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:54.473162 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-30715221-b77a-48fe-8d9d-d6b7e59e4638 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:54.474337 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-30715221-b77a-48fe-8d9d-d6b7e59e4638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:54.543368 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 518/991] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:54 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:54.544252 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30715221-b77a-48fe-8d9d-d6b7e59e4638 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:54.549079 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5619c4-0f04-47db-b4a3-2c674f3b9578 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:54.566608 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:54.590389 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01ab39-4d97-4304-816e-9f1f0a02e4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:54.591072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01ab39-4d97-4304-816e-9f1f0a02e4c2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:54.591376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01ab39-4d97-4304-816e-9f1f0a02e4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:54.591738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01ab39-4d97-4304-816e-9f1f0a02e4c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:54.592697 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01ab39-4d97-4304-816e-9f1f0a02e4c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:54.593080 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01ab39-4d97-4304-816e-9f1f0a02e4c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:54.594945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01ab39-4d97-4304-816e-9f1f0a02e4c2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:54.612626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 474/992] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:20:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:54.613632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01ab39-4d97-4304-816e-9f1f0a02e4c2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:54.650115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:54.658732 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30715221-b77a-48fe-8d9d-d6b7e59e4638 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:54.696735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5dab04-701b-4d80-9a2a-9a07ec2a30ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:54.697792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5dab04-701b-4d80-9a2a-9a07ec2a30ea None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:54.698389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5dab04-701b-4d80-9a2a-9a07ec2a30ea None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:54.698699 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5dab04-701b-4d80-9a2a-9a07ec2a30ea None admin] PATH_INFO: `/v3/services/compute` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:54.699285 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5dab04-701b-4d80-9a2a-9a07ec2a30ea None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:54.701516 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5dab04-701b-4d80-9a2a-9a07ec2a30ea None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:54.713166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-1e5dab04-701b-4d80-9a2a-9a07ec2a30ea None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:20:54.713166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:54.713166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:54.713166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:54.713166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:54.713166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:54.713166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:54.713166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:54.713166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:20:54.713166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:54.713166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:54.713166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:54.713166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:54.713166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:54.713166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:54.713166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:54.713166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:54.714705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:54.714705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:54.714705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:54.714705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:54.714705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:54.714705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:54.714705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:54.714705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:54.714705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:54.714705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:54.714705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:54.714705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:54.714705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:54.714705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:54.714705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:54.714705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:54.715605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:54.715605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:54.715605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:54.715605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:54.715605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:54.715605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:54.715605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:54.715605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:54.715605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:54.715605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:54.715605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:54.715605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:54.715605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:20:54.715605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:54.715605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 519/993] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:54.720655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5dab04-701b-4d80-9a2a-9a07ec2a30ea None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:54.723436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01ab39-4d97-4304-816e-9f1f0a02e4c2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:54.757177 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ea3c6-f4f8-4072-85bd-d7dd33f3251d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:54.757776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ea3c6-f4f8-4072-85bd-d7dd33f3251d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:54.758051 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ea3c6-f4f8-4072-85bd-d7dd33f3251d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:54.758319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ea3c6-f4f8-4072-85bd-d7dd33f3251d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:54.759031 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ea3c6-f4f8-4072-85bd-d7dd33f3251d None admin] RBAC: Adding query filter params (role.id=8ece00b091b347179365b72919fdd7dc, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=61709ccc1e244185b252179451cd860f) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:54.759332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ea3c6-f4f8-4072-85bd-d7dd33f3251d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:54.760821 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ea3c6-f4f8-4072-85bd-d7dd33f3251d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:54.766510 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 475/994] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:20:54 2022] GET /identity/v3/role_assignments?user.id=61709ccc1e244185b252179451cd860f&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=8ece00b091b347179365b72919fdd7dc => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:54.823112 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ea3c6-f4f8-4072-85bd-d7dd33f3251d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:54.826358 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:54.847619 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5dab04-701b-4d80-9a2a-9a07ec2a30ea None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:54.887049 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505152aa-caa3-4e62-8254-965cf85be3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:54.887676 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505152aa-caa3-4e62-8254-965cf85be3ed None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:54.888480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505152aa-caa3-4e62-8254-965cf85be3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:54.888786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505152aa-caa3-4e62-8254-965cf85be3ed None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:54.889438 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505152aa-caa3-4e62-8254-965cf85be3ed None admin] RBAC: Adding query filter params (name=compute) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:54.889742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505152aa-caa3-4e62-8254-965cf85be3ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:54.891135 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505152aa-caa3-4e62-8254-965cf85be3ed None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:54.902699 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 520/995] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 29 20:20:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:54.903670 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505152aa-caa3-4e62-8254-965cf85be3ed None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:54.947258 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:54.953123 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ea3c6-f4f8-4072-85bd-d7dd33f3251d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:54.990263 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea474a4-47d2-4bf6-af09-0afaea3b365f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:54.990906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea474a4-47d2-4bf6-af09-0afaea3b365f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:54.991185 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea474a4-47d2-4bf6-af09-0afaea3b365f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:54.991456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea474a4-47d2-4bf6-af09-0afaea3b365f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:54.992061 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea474a4-47d2-4bf6-af09-0afaea3b365f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:54.993423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea474a4-47d2-4bf6-af09-0afaea3b365f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:55.002617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-5ea474a4-47d2-4bf6-af09-0afaea3b365f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:20:55.002617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:55.002617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:55.002617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:55.002617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:55.002617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:55.002617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:55.002617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:55.002617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:20:55.002617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:55.002617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:55.002617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:55.002617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:55.002617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:55.002617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:55.002617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:55.002617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:55.003755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:55.003755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:55.003755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:55.003755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:55.003755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:55.003755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:55.003755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:55.003755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:55.003755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:55.003755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:55.003755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:55.003755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:55.003755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:55.003755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:55.003755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:55.003755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:55.004612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:55.004612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:55.004612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:55.004612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:55.004612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:55.004612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:55.004612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:55.004612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:55.004612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:55.004612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:55.004612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:55.004612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:55.004612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 29 20:20:55.004612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:20:55.004612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 476/996] 10.222.0.18 () {56 vars in 1126 bytes} [Mon Aug 29 20:20:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:55.009697 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea474a4-47d2-4bf6-af09-0afaea3b365f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:55.029326 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505152aa-caa3-4e62-8254-965cf85be3ed None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:55.066254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a832342-7ad3-4121-afcb-83b181ce86df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:55.066877 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a832342-7ad3-4121-afcb-83b181ce86df None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:55.067162 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a832342-7ad3-4121-afcb-83b181ce86df None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:55.067438 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a832342-7ad3-4121-afcb-83b181ce86df None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:55.068110 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a832342-7ad3-4121-afcb-83b181ce86df None admin] RBAC: Adding query filter params (type=compute) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:55.068414 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a832342-7ad3-4121-afcb-83b181ce86df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:55.069769 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a832342-7ad3-4121-afcb-83b181ce86df None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:55.075456 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 521/997] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 29 20:20:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:55.081255 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a832342-7ad3-4121-afcb-83b181ce86df None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:55.093885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:55.119405 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea474a4-47d2-4bf6-af09-0afaea3b365f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:55.158437 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75391d1-35cd-456d-a487-217ffd7f75ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:55.159044 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75391d1-35cd-456d-a487-217ffd7f75ba None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:55.159324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75391d1-35cd-456d-a487-217ffd7f75ba None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:55.159599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75391d1-35cd-456d-a487-217ffd7f75ba None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:55.160270 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75391d1-35cd-456d-a487-217ffd7f75ba None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:55.160576 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75391d1-35cd-456d-a487-217ffd7f75ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:55.161967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75391d1-35cd-456d-a487-217ffd7f75ba None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:55.167703 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 477/998] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Aug 29 20:20:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:55.173229 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75391d1-35cd-456d-a487-217ffd7f75ba None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:55.188199 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a832342-7ad3-4121-afcb-83b181ce86df None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:55.204556 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:55.228567 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874e420-a550-42d4-9740-1f98968be663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:55.229470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874e420-a550-42d4-9740-1f98968be663 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:55.229885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874e420-a550-42d4-9740-1f98968be663 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:55.230304 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874e420-a550-42d4-9740-1f98968be663 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:55.231157 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f874e420-a550-42d4-9740-1f98968be663 None admin] RBAC: Adding query filter params (interface=public, service_id=0df1ab88b2014f8b817fa77213e5dd7b, region_id=RegionOne) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:55.231465 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f874e420-a550-42d4-9740-1f98968be663 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:55.232866 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f874e420-a550-42d4-9740-1f98968be663 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:55.237875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 522/999] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Aug 29 20:20:55 2022] GET /identity/v3/endpoints?service_id=0df1ab88b2014f8b817fa77213e5dd7b&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:55.243254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874e420-a550-42d4-9740-1f98968be663 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:55.287750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75391d1-35cd-456d-a487-217ffd7f75ba None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:55.327684 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324390d-2278-4e24-b52e-3a3890ad402b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:55.328265 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324390d-2278-4e24-b52e-3a3890ad402b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:55.328545 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324390d-2278-4e24-b52e-3a3890ad402b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:55.328819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324390d-2278-4e24-b52e-3a3890ad402b None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:55.329450 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324390d-2278-4e24-b52e-3a3890ad402b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:55.329751 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324390d-2278-4e24-b52e-3a3890ad402b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:55.331110 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324390d-2278-4e24-b52e-3a3890ad402b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:55.335316 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:55.340937 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 478/1000] 10.222.0.18 () {56 vars in 1122 bytes} [Mon Aug 29 20:20:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:55.341817 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324390d-2278-4e24-b52e-3a3890ad402b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:55.370553 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874e420-a550-42d4-9740-1f98968be663 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:55.406196 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99dbd2-3cca-4292-94e3-3ed9a01be17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:55.407523 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99dbd2-3cca-4292-94e3-3ed9a01be17f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:55.407809 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99dbd2-3cca-4292-94e3-3ed9a01be17f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:55.408123 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99dbd2-3cca-4292-94e3-3ed9a01be17f None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:55.408733 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d99dbd2-3cca-4292-94e3-3ed9a01be17f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:55.410077 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d99dbd2-3cca-4292-94e3-3ed9a01be17f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:55.427964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 523/1001] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Aug 29 20:20:55 2022] GET /identity/v3/services => generated 1727 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:20:55.447516 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:55.459697 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324390d-2278-4e24-b52e-3a3890ad402b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:55.497369 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79133022-6e35-4760-9d8d-1a960b1314d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:55.497997 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79133022-6e35-4760-9d8d-1a960b1314d0 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:55.498274 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79133022-6e35-4760-9d8d-1a960b1314d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:55.498544 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79133022-6e35-4760-9d8d-1a960b1314d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:55.499232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79133022-6e35-4760-9d8d-1a960b1314d0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:55.500679 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79133022-6e35-4760-9d8d-1a960b1314d0 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:55.519913 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 479/1002] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:20:55 2022] POST /identity/v3/endpoints => generated 325 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:20:56.731073 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04370b63-a1c0-4ea4-b668-7e405e4bf596 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:56.731632 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04370b63-a1c0-4ea4-b668-7e405e4bf596 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:56.731632 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04370b63-a1c0-4ea4-b668-7e405e4bf596 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:56.733176 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 524/1003] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:56.738132 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae94331-c68c-4f1f-93a8-253ce9aaf42a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:56.738549 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae94331-c68c-4f1f-93a8-253ce9aaf42a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:56.738723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae94331-c68c-4f1f-93a8-253ce9aaf42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:56.757226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1ae94331-c68c-4f1f-93a8-253ce9aaf42a None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:56.758411 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1ae94331-c68c-4f1f-93a8-253ce9aaf42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:56.833891 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 480/1004] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:56 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:57.101140 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04370b63-a1c0-4ea4-b668-7e405e4bf596 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:57.209450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04370b63-a1c0-4ea4-b668-7e405e4bf596 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:57.243684 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc5efd-0872-47b0-b62d-725c7ed7c28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:57.244346 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc5efd-0872-47b0-b62d-725c7ed7c28b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:57.244659 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc5efd-0872-47b0-b62d-725c7ed7c28b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:57.244946 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc5efd-0872-47b0-b62d-725c7ed7c28b None admin] PATH_INFO: `/v3/services/image` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:57.245530 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc5efd-0872-47b0-b62d-725c7ed7c28b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:57.246891 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc5efd-0872-47b0-b62d-725c7ed7c28b None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:57.249549 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:57.257214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-25cc5efd-0872-47b0-b62d-725c7ed7c28b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:20:57.257214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:57.257214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:57.257214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:57.257214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:57.257214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:57.257214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:57.257214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:57.257214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:20:57.257214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:57.257214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:57.257214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:57.257214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:57.257214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:57.257214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:57.257214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:57.257214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:57.258282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:57.258282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:57.258282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:57.258282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:57.258282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:57.258282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:57.258282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:57.258282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:57.258282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:57.258282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:57.258282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:57.258282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:57.258282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:57.258282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:57.258282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:57.258282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:57.259194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:57.259194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:57.259194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:57.259194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:57.259194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:57.259194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:57.259194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:57.259194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:57.259194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:57.259194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:57.259194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:57.259194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:57.259194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:20:57.259194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:57.259194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 525/1005] 10.222.0.18 () {56 vars in 1105 bytes} [Mon Aug 29 20:20:57 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:20:57.267457 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae94331-c68c-4f1f-93a8-253ce9aaf42a None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:57.358676 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:57.367976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e93c5-ef20-4247-9166-5159e47b5c9d None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:57.368276 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e93c5-ef20-4247-9166-5159e47b5c9d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:57.368541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e93c5-ef20-4247-9166-5159e47b5c9d None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:57.369917 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 526/1006] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:20:57.374943 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845cfba-4f15-4ec5-b26b-12445cccd51b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:57.375233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845cfba-4f15-4ec5-b26b-12445cccd51b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:57.375509 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5845cfba-4f15-4ec5-b26b-12445cccd51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:57.386420 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae94331-c68c-4f1f-93a8-253ce9aaf42a None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:57.395813 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-5845cfba-4f15-4ec5-b26b-12445cccd51b None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:57.397036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5845cfba-4f15-4ec5-b26b-12445cccd51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:57.424555 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d1f49-f7e4-4cef-a948-3a7759324d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:57.425166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d1f49-f7e4-4cef-a948-3a7759324d02 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:57.425449 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d1f49-f7e4-4cef-a948-3a7759324d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:57.425714 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d1f49-f7e4-4cef-a948-3a7759324d02 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:57.426340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d1f49-f7e4-4cef-a948-3a7759324d02 None admin] RBAC: Adding query filter params (name=image) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:57.426640 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d1f49-f7e4-4cef-a948-3a7759324d02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:57.428028 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d1f49-f7e4-4cef-a948-3a7759324d02 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:57.440091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 481/1007] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:57.441146 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d1f49-f7e4-4cef-a948-3a7759324d02 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:57.476146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 527/1008] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:57 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:20:57.477524 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad14a9-17e6-498b-99b1-517c6782bcda None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:57.477814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad14a9-17e6-498b-99b1-517c6782bcda None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:57.478083 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad14a9-17e6-498b-99b1-517c6782bcda None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:57.479874 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 528/1009] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:57.484975 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1cb34-3dba-4878-9c92-8a45a06587b9 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:57.485321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1cb34-3dba-4878-9c92-8a45a06587b9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:57.485596 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1cb34-3dba-4878-9c92-8a45a06587b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:57.504244 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-25f1cb34-3dba-4878-9c92-8a45a06587b9 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:57.505425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-25f1cb34-3dba-4878-9c92-8a45a06587b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:57.561753 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d1f49-f7e4-4cef-a948-3a7759324d02 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:57.579251 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 529/1010] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:57 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:20:57.596138 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08913889-192a-4112-919b-781a91b964d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:57.596749 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08913889-192a-4112-919b-781a91b964d8 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:57.597031 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08913889-192a-4112-919b-781a91b964d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:57.597361 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08913889-192a-4112-919b-781a91b964d8 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:57.597997 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08913889-192a-4112-919b-781a91b964d8 None admin] RBAC: Adding query filter params (type=image) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:57.598304 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08913889-192a-4112-919b-781a91b964d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:57.600038 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08913889-192a-4112-919b-781a91b964d8 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:57.612379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 482/1011] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:57 2022] GET /identity/v3/services?type=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:20:57.747356 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1cb34-3dba-4878-9c92-8a45a06587b9 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:57.818012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:57.852866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08913889-192a-4112-919b-781a91b964d8 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:57.857268 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:57.863902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1cb34-3dba-4878-9c92-8a45a06587b9 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:57.899315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265637b-a25d-4ffc-b96c-3bb980c9f5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:57.900345 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265637b-a25d-4ffc-b96c-3bb980c9f5de None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:57.900622 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265637b-a25d-4ffc-b96c-3bb980c9f5de None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:57.900922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265637b-a25d-4ffc-b96c-3bb980c9f5de None admin] PATH_INFO: `/v3/services/compute` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:57.901516 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265637b-a25d-4ffc-b96c-3bb980c9f5de None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:57.902955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265637b-a25d-4ffc-b96c-3bb980c9f5de None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:57.911976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-9265637b-a25d-4ffc-b96c-3bb980c9f5de None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:20:57.911976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:57.911976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:20:57.911976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:20:57.911976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:20:57.911976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:20:57.911976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:20:57.911976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:57.911976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:20:57.911976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:57.911976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:20:57.911976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:57.911976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:20:57.911976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:20:57.911976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:20:57.911976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:20:57.911976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:20:57.913063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:20:57.913063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:20:57.913063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:20:57.913063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:20:57.913063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:20:57.913063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:20:57.913063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:20:57.913063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:20:57.913063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:20:57.913063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:20:57.913063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:20:57.913063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:20:57.913063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:20:57.913063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:20:57.913063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:20:57.913063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:20:57.914147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:20:57.914147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:20:57.914147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:20:57.914147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:20:57.914147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:20:57.914147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:20:57.914147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:20:57.914147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:20:57.914147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:20:57.914147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:20:57.914147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:20:57.914147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:20:57.914147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 29 20:20:57.914147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:20:57.914147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 530/1012] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:20:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:20:57.919557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265637b-a25d-4ffc-b96c-3bb980c9f5de None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:57.960774 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08913889-192a-4112-919b-781a91b964d8 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:57.996917 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb68233-bf1b-4c83-b87f-4f3bb2a1ef8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:57.997820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb68233-bf1b-4c83-b87f-4f3bb2a1ef8f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:57.998097 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb68233-bf1b-4c83-b87f-4f3bb2a1ef8f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:57.998368 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb68233-bf1b-4c83-b87f-4f3bb2a1ef8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:57.999457 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb68233-bf1b-4c83-b87f-4f3bb2a1ef8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:58.000901 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb68233-bf1b-4c83-b87f-4f3bb2a1ef8f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:58.002464 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 483/1013] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:20:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:58.008184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb68233-bf1b-4c83-b87f-4f3bb2a1ef8f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:58.033502 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265637b-a25d-4ffc-b96c-3bb980c9f5de None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:58.060619 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:58.076814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bed40-31a2-479f-84b8-47890fc50f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:58.077839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bed40-31a2-479f-84b8-47890fc50f15 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:58.078373 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bed40-31a2-479f-84b8-47890fc50f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:58.078658 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bed40-31a2-479f-84b8-47890fc50f15 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:58.079292 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4bed40-31a2-479f-84b8-47890fc50f15 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:58.079601 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4bed40-31a2-479f-84b8-47890fc50f15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:58.081825 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4bed40-31a2-479f-84b8-47890fc50f15 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:58.085959 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:58.094489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 531/1014] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 29 20:20:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:58.095363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bed40-31a2-479f-84b8-47890fc50f15 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:58.126943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb68233-bf1b-4c83-b87f-4f3bb2a1ef8f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:58.164119 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027f953-5649-448c-8b0c-a1a3f7f0fe13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:58.164883 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027f953-5649-448c-8b0c-a1a3f7f0fe13 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:58.165170 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027f953-5649-448c-8b0c-a1a3f7f0fe13 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:58.165426 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027f953-5649-448c-8b0c-a1a3f7f0fe13 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:58.166063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027f953-5649-448c-8b0c-a1a3f7f0fe13 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:58.168025 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027f953-5649-448c-8b0c-a1a3f7f0fe13 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:58.196306 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 484/1015] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 29 20:20:58 2022] POST /identity/v3/projects => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:20:58.205731 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bed40-31a2-479f-84b8-47890fc50f15 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:58.254925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da033cf3-1899-4aa7-84ce-8ff17b046da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:58.255552 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da033cf3-1899-4aa7-84ce-8ff17b046da4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:58.255833 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da033cf3-1899-4aa7-84ce-8ff17b046da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:58.256137 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da033cf3-1899-4aa7-84ce-8ff17b046da4 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:58.256782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da033cf3-1899-4aa7-84ce-8ff17b046da4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:20:58.257085 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da033cf3-1899-4aa7-84ce-8ff17b046da4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:58.258464 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da033cf3-1899-4aa7-84ce-8ff17b046da4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:58.268723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 532/1016] 10.222.0.18 () {56 vars in 1112 bytes} [Mon Aug 29 20:20:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:20:58.269998 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027f953-5649-448c-8b0c-a1a3f7f0fe13 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:20:58.389172 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027f953-5649-448c-8b0c-a1a3f7f0fe13 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:20:58.413075 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:20:58.425598 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1a301-1e7e-4165-ba17-57656c5f5e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:20:58.426201 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1a301-1e7e-4165-ba17-57656c5f5e68 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:58.426472 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1a301-1e7e-4165-ba17-57656c5f5e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:58.426736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1a301-1e7e-4165-ba17-57656c5f5e68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:58.427406 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1a301-1e7e-4165-ba17-57656c5f5e68 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:20:58.428795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1a301-1e7e-4165-ba17-57656c5f5e68 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:20:58.455966 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 485/1017] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:20:58 2022] POST /identity/v3/endpoints => generated 334 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:20:59.565355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06810be-80e5-4415-9c09-fbc450bfe775 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:59.565849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06810be-80e5-4415-9c09-fbc450bfe775 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:59.565849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06810be-80e5-4415-9c09-fbc450bfe775 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:59.567975 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 533/1018] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:20:59.572363 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682af88-ab2f-44da-beed-fc03c4402e65 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:20:59.573299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682af88-ab2f-44da-beed-fc03c4402e65 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:20:59.573393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682af88-ab2f-44da-beed-fc03c4402e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:20:59.599356 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e682af88-ab2f-44da-beed-fc03c4402e65 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:20:59.600583 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e682af88-ab2f-44da-beed-fc03c4402e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:20:59.683974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 486/1019] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:20:59 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:20:59.968329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06810be-80e5-4415-9c09-fbc450bfe775 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:00.027656 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:00.083755 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06810be-80e5-4415-9c09-fbc450bfe775 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:00.129228 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffea5ee-d341-48cd-8842-7d4252033161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:00.129834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffea5ee-d341-48cd-8842-7d4252033161 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:00.130117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffea5ee-d341-48cd-8842-7d4252033161 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:00.130391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffea5ee-d341-48cd-8842-7d4252033161 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:00.131033 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffea5ee-d341-48cd-8842-7d4252033161 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:00.132438 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffea5ee-d341-48cd-8842-7d4252033161 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:00.151763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 534/1020] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 29 20:20:59 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:00.272926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235abe20-3ce1-469b-a90d-7abbdd7df4e5 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:00.273288 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235abe20-3ce1-469b-a90d-7abbdd7df4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:00.273482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235abe20-3ce1-469b-a90d-7abbdd7df4e5 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:00.275065 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 487/1021] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:00.280783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ad887-5915-40c1-a2c2-f4d044275e44 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:00.281107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ad887-5915-40c1-a2c2-f4d044275e44 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:00.281357 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ad887-5915-40c1-a2c2-f4d044275e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:00.310425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-3a7ad887-5915-40c1-a2c2-f4d044275e44 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:00.311630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3a7ad887-5915-40c1-a2c2-f4d044275e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:00.394487 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 535/1022] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:00.485904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce537ac-c614-4b2b-846e-a12c77e855f7 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:00.486244 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce537ac-c614-4b2b-846e-a12c77e855f7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:00.486459 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce537ac-c614-4b2b-846e-a12c77e855f7 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:00.487992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 488/1023] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:00.493037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1c4d8-704c-463f-a098-54cb9e3821c7 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:00.493366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1c4d8-704c-463f-a098-54cb9e3821c7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:00.493596 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1c4d8-704c-463f-a098-54cb9e3821c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:00.515700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-7dc1c4d8-704c-463f-a098-54cb9e3821c7 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:00.516922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7dc1c4d8-704c-463f-a098-54cb9e3821c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:00.552546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:00.598625 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 536/1024] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:00.677688 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce537ac-c614-4b2b-846e-a12c77e855f7 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:00.789321 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce537ac-c614-4b2b-846e-a12c77e855f7 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:00.799553 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:00.831074 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f55165-8408-465c-93f5-a9c4ff94e329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:00.831597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f55165-8408-465c-93f5-a9c4ff94e329 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:00.831880 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f55165-8408-465c-93f5-a9c4ff94e329 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:00.832208 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f55165-8408-465c-93f5-a9c4ff94e329 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:00.833346 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f55165-8408-465c-93f5-a9c4ff94e329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:00.834777 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f55165-8408-465c-93f5-a9c4ff94e329 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:00.836679 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 489/1025] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:21:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:00.842602 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1c4d8-704c-463f-a098-54cb9e3821c7 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:00.869943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f55165-8408-465c-93f5-a9c4ff94e329 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:00.970430 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1c4d8-704c-463f-a098-54cb9e3821c7 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:00.993837 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f55165-8408-465c-93f5-a9c4ff94e329 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:01.015132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490313f1-4c5e-468d-9d3b-1462cbef4684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:01.016155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490313f1-4c5e-468d-9d3b-1462cbef4684 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:01.016445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490313f1-4c5e-468d-9d3b-1462cbef4684 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:01.016722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490313f1-4c5e-468d-9d3b-1462cbef4684 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:01.017397 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490313f1-4c5e-468d-9d3b-1462cbef4684 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:01.018840 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490313f1-4c5e-468d-9d3b-1462cbef4684 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:01.038888 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0c026-00fa-4544-9778-012ff826ac6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:01.040151 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0c026-00fa-4544-9778-012ff826ac6d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:01.040431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0c026-00fa-4544-9778-012ff826ac6d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:01.040697 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0c026-00fa-4544-9778-012ff826ac6d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:01.052239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 537/1026] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 29 20:21:00 2022] POST /identity/v3/users => generated 285 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:01.059022 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0c026-00fa-4544-9778-012ff826ac6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:21:01.064706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-2eb0c026-00fa-4544-9778-012ff826ac6d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:21:01.064706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:01.064706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:01.064706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:01.064706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:01.064706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:01.064706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:01.064706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:01.064706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:01.064706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:01.064706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:01.064706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:01.064706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:21:01.064706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:21:01.064706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:21:01.064706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:21:01.065807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:21:01.065807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:21:01.065807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:21:01.065807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:21:01.065807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:01.065807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:01.065807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:01.065807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:01.065807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:01.065807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:01.065807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:01.065807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:01.065807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:01.065807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:01.065807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:01.065807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:01.066693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:01.066693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:01.066693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:01.066693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:01.066693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:21:01.066693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:21:01.066693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:21:01.066693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:21:01.066693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:21:01.066693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:21:01.066693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:21:01.066693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:21:01.066693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 490/1027] 10.222.0.18 () {56 vars in 1120 bytes} [Mon Aug 29 20:21:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:21:01.072207 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490313f1-4c5e-468d-9d3b-1462cbef4684 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:01.111483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:01.214805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490313f1-4c5e-468d-9d3b-1462cbef4684 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:01.253984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42208428-d4ed-4f48-ba7b-611a5cea1c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:01.254608 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42208428-d4ed-4f48-ba7b-611a5cea1c52 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:01.254880 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42208428-d4ed-4f48-ba7b-611a5cea1c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:01.255244 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42208428-d4ed-4f48-ba7b-611a5cea1c52 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:01.255863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42208428-d4ed-4f48-ba7b-611a5cea1c52 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:01.256254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42208428-d4ed-4f48-ba7b-611a5cea1c52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:01.257586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42208428-d4ed-4f48-ba7b-611a5cea1c52 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:01.269136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 538/1028] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Aug 29 20:21:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:01.275126 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0c026-00fa-4544-9778-012ff826ac6d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:01.318630 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:01.389392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0c026-00fa-4544-9778-012ff826ac6d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:01.426899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d5862-bb18-4779-8a82-cf06cba123d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:01.427538 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d5862-bb18-4779-8a82-cf06cba123d4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:01.427817 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d5862-bb18-4779-8a82-cf06cba123d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:01.428117 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d5862-bb18-4779-8a82-cf06cba123d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:01.438347 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d5862-bb18-4779-8a82-cf06cba123d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:01.439859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d5862-bb18-4779-8a82-cf06cba123d4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:01.455476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-f73d5862-bb18-4779-8a82-cf06cba123d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:21:01.455476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:01.455476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:01.455476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:01.455476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:01.455476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:01.455476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:01.455476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:01.455476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:01.455476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:01.455476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:01.455476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:01.455476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:21:01.455476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:21:01.455476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:21:01.455476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:21:01.456531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:01.456531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:01.456531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:01.456531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:01.456531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:01.456531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:01.456531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:01.456531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:01.456531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:01.456531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:01.456531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:01.456531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:01.456531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:01.456531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:01.456531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:01.456531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:01.456531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:21:01.457430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:21:01.457430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:21:01.457430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:21:01.457430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 491/1029] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:21:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:21:01.464290 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42208428-d4ed-4f48-ba7b-611a5cea1c52 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:01.555039 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:01.572403 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42208428-d4ed-4f48-ba7b-611a5cea1c52 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:01.616572 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda5c9e-e094-43f0-a663-8ff4f2dcc516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:01.617223 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda5c9e-e094-43f0-a663-8ff4f2dcc516 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:01.617481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda5c9e-e094-43f0-a663-8ff4f2dcc516 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:01.617732 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda5c9e-e094-43f0-a663-8ff4f2dcc516 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:01.618386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda5c9e-e094-43f0-a663-8ff4f2dcc516 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:01.618693 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda5c9e-e094-43f0-a663-8ff4f2dcc516 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:01.620159 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda5c9e-e094-43f0-a663-8ff4f2dcc516 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:01.638333 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 539/1030] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:01.639698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d5862-bb18-4779-8a82-cf06cba123d4 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:01.734123 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:01.752477 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d5862-bb18-4779-8a82-cf06cba123d4 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:01.807143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e317efa-6170-4192-826c-77b63d4ecfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:01.810308 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e317efa-6170-4192-826c-77b63d4ecfb5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:01.810592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e317efa-6170-4192-826c-77b63d4ecfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:01.810869 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e317efa-6170-4192-826c-77b63d4ecfb5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:01.820185 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e317efa-6170-4192-826c-77b63d4ecfb5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:01.821635 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e317efa-6170-4192-826c-77b63d4ecfb5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:01.833612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-7e317efa-6170-4192-826c-77b63d4ecfb5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:21:01.833612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:01.833612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:01.833612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:01.833612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:01.833612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:01.833612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:01.833612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:01.833612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:01.833612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:01.833612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:01.833612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:01.833612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:21:01.833612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:21:01.833612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:21:01.833612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:21:01.834692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:01.834692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:01.834692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:21:01.834692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:21:01.834692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:21:01.834692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:21:01.834692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:01.834692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:01.834692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:01.834692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:01.834692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:01.834692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:01.834692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:01.834692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:01.834692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:01.834692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:01.834692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:01.834692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:01.835636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:01.835636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:01.835636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:21:01.835636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:21:01.835636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:21:01.835636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:21:01.835636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:21:01.835636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:21:01.835636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:21:01.835636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:21:01.840192 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 492/1031] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Aug 29 20:21:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:21:01.841454 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda5c9e-e094-43f0-a663-8ff4f2dcc516 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:01.957766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda5c9e-e094-43f0-a663-8ff4f2dcc516 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:01.995784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3815f1-f070-400b-aea3-3aa26ef1907a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:01.996774 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3815f1-f070-400b-aea3-3aa26ef1907a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:01.997288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3815f1-f070-400b-aea3-3aa26ef1907a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:01.997932 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3815f1-f070-400b-aea3-3aa26ef1907a None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:01.999012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3815f1-f070-400b-aea3-3aa26ef1907a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:01.999524 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3815f1-f070-400b-aea3-3aa26ef1907a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:02.001177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3815f1-f070-400b-aea3-3aa26ef1907a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:02.021218 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 540/1032] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Aug 29 20:21:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:02.026793 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e317efa-6170-4192-826c-77b63d4ecfb5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:02.146134 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e317efa-6170-4192-826c-77b63d4ecfb5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:02.167622 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6424e6b-4ecc-4703-b6db-75018a654c0c None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:02.168284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6424e6b-4ecc-4703-b6db-75018a654c0c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:02.168780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6424e6b-4ecc-4703-b6db-75018a654c0c None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:02.170309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 541/1033] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:21:02.177519 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f1ea7-0488-40e1-8ec0-2a29021f0a8c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:02.178472 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f1ea7-0488-40e1-8ec0-2a29021f0a8c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:02.178960 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f1ea7-0488-40e1-8ec0-2a29021f0a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:02.186644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41344909-d842-4e19-aae0-d873d498d746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:02.187253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41344909-d842-4e19-aae0-d873d498d746 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:02.187387 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:02.187535 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41344909-d842-4e19-aae0-d873d498d746 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:02.187814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41344909-d842-4e19-aae0-d873d498d746 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:02.201280 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d94f1ea7-0488-40e1-8ec0-2a29021f0a8c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:02.202875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d94f1ea7-0488-40e1-8ec0-2a29021f0a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:02.207756 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:02.269306 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41344909-d842-4e19-aae0-d873d498d746 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:02.271907 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41344909-d842-4e19-aae0-d873d498d746 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:02.276367 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 542/1034] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:02.397389 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 493/1035] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:21:02 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:21:02.399097 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f1ea7-0488-40e1-8ec0-2a29021f0a8c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:02.514527 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f1ea7-0488-40e1-8ec0-2a29021f0a8c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:02.560672 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41344909-d842-4e19-aae0-d873d498d746 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:02.562998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b785c9-9f85-4d02-9b70-472f90d9c4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:02.563645 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b785c9-9f85-4d02-9b70-472f90d9c4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:02.563942 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b785c9-9f85-4d02-9b70-472f90d9c4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:02.564236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b785c9-9f85-4d02-9b70-472f90d9c4f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:02.567302 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:02.585088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b785c9-9f85-4d02-9b70-472f90d9c4f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:02.586553 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b785c9-9f85-4d02-9b70-472f90d9c4f4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:02.601792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-b2b785c9-9f85-4d02-9b70-472f90d9c4f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:21:02.601792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:02.601792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:02.601792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:02.601792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:02.601792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:02.601792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:02.601792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:02.601792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:02.601792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:02.601792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:02.601792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:02.601792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:21:02.601792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:21:02.601792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:21:02.601792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:21:02.602760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:02.602760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:02.602760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:02.602760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:02.602760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:02.602760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:02.602760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:02.602760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:02.602760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:02.602760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:02.602760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:02.602760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:02.602760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:02.602760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:02.602760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:02.602760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:02.602760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:21:02.604247 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:21:02.604247 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:21:02.604247 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:21:02.604247 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 543/1036] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:21:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:21:02.610128 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b785c9-9f85-4d02-9b70-472f90d9c4f4 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:02.625033 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:02.684740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41344909-d842-4e19-aae0-d873d498d746 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:02.721394 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2eb33-7e8d-4187-b0f0-f733dc984726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:02.722025 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2eb33-7e8d-4187-b0f0-f733dc984726 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:02.722300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2eb33-7e8d-4187-b0f0-f733dc984726 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:02.722630 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2eb33-7e8d-4187-b0f0-f733dc984726 None admin] PATH_INFO: `/v3/services/image` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:02.723212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2eb33-7e8d-4187-b0f0-f733dc984726 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:02.724600 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2eb33-7e8d-4187-b0f0-f733dc984726 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:02.734076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-91d2eb33-7e8d-4187-b0f0-f733dc984726 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:21:02.734076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:02.734076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:21:02.734076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:21:02.734076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:21:02.734076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:21:02.734076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:21:02.734076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:21:02.734076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:21:02.734076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:21:02.734076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:21:02.734076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:21:02.734076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:02.734076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:02.734076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:02.734076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:02.734076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:02.735610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:02.735610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:02.735610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:02.735610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:02.735610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:02.735610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:02.735610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:21:02.735610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:21:02.735610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:21:02.735610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:21:02.735610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:02.735610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:02.735610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:02.735610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:02.735610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:02.735610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:02.736572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:02.736572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:02.736572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:02.736572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:02.736572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:02.736572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:02.736572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:02.736572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:02.736572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:02.736572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:02.736572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:21:02.736572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:21:02.736572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:21:02.736572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:21:02.736572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 494/1037] 10.222.0.18 () {56 vars in 1105 bytes} [Mon Aug 29 20:21:02 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:21:02.739623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b785c9-9f85-4d02-9b70-472f90d9c4f4 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:02.741168 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2eb33-7e8d-4187-b0f0-f733dc984726 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:02.776322 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54724cb-fe5c-471e-9fec-e8a72d5a84cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:02.777201 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54724cb-fe5c-471e-9fec-e8a72d5a84cb None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:02.777488 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54724cb-fe5c-471e-9fec-e8a72d5a84cb None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:02.777957 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54724cb-fe5c-471e-9fec-e8a72d5a84cb None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:02.778410 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54724cb-fe5c-471e-9fec-e8a72d5a84cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:02.778760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54724cb-fe5c-471e-9fec-e8a72d5a84cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:02.780111 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54724cb-fe5c-471e-9fec-e8a72d5a84cb None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:02.794303 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 544/1038] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:21:02.800052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54724cb-fe5c-471e-9fec-e8a72d5a84cb None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:02.851854 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:02.854766 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2eb33-7e8d-4187-b0f0-f733dc984726 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:02.865604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:02.892567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13486a3-2c1d-4b1a-8f58-13b3b3b8d97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:02.893180 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13486a3-2c1d-4b1a-8f58-13b3b3b8d97a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:02.893478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13486a3-2c1d-4b1a-8f58-13b3b3b8d97a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:02.893757 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13486a3-2c1d-4b1a-8f58-13b3b3b8d97a None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:02.894390 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13486a3-2c1d-4b1a-8f58-13b3b3b8d97a None admin] RBAC: Adding query filter params (name=image) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:02.894692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13486a3-2c1d-4b1a-8f58-13b3b3b8d97a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:02.896101 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13486a3-2c1d-4b1a-8f58-13b3b3b8d97a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:02.901573 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 495/1039] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:21:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:21:02.907470 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13486a3-2c1d-4b1a-8f58-13b3b3b8d97a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:02.915653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54724cb-fe5c-471e-9fec-e8a72d5a84cb None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:02.955440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991bfcc-d23b-4641-a6aa-5aad9314199a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:02.956531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991bfcc-d23b-4641-a6aa-5aad9314199a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:02.956823 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991bfcc-d23b-4641-a6aa-5aad9314199a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:02.957410 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991bfcc-d23b-4641-a6aa-5aad9314199a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:02.967399 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991bfcc-d23b-4641-a6aa-5aad9314199a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:02.970320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991bfcc-d23b-4641-a6aa-5aad9314199a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:02.984330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-e991bfcc-d23b-4641-a6aa-5aad9314199a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:21:02.984330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:02.984330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:02.984330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:02.984330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:02.984330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:02.984330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:02.984330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:02.984330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:02.984330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:02.984330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:02.984330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:02.984330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:21:02.984330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:21:02.984330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:21:02.985310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:21:02.985310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:02.985310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:02.985310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:02.985310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:02.985310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:02.985310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:02.985310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:02.985310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:02.985310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:02.985310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:02.985310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:02.985310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:02.985310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:02.985310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:02.985310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:02.985310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:02.986191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:21:02.986191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:21:02.986191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:21:02.986191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:21:02.986191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:21:02.986191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:21:02.986191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:21:02.986191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:21:02.986191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 545/1040] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:21:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:21:02.991144 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991bfcc-d23b-4641-a6aa-5aad9314199a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:03.017337 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13486a3-2c1d-4b1a-8f58-13b3b3b8d97a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:03.054623 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29474a0c-514d-4dac-b437-2022ee7921fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:03.055226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29474a0c-514d-4dac-b437-2022ee7921fa None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:03.055581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29474a0c-514d-4dac-b437-2022ee7921fa None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:03.055759 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29474a0c-514d-4dac-b437-2022ee7921fa None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:03.056421 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29474a0c-514d-4dac-b437-2022ee7921fa None admin] RBAC: Adding query filter params (type=image) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:03.056773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29474a0c-514d-4dac-b437-2022ee7921fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:03.058134 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29474a0c-514d-4dac-b437-2022ee7921fa None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:03.068585 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 496/1041] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:21:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:21:03.069456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29474a0c-514d-4dac-b437-2022ee7921fa None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:03.104304 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991bfcc-d23b-4641-a6aa-5aad9314199a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:03.111870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:03.114913 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:03.146024 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025d5e6-3b94-4aca-b978-a7a87e5aadfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:03.146760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025d5e6-3b94-4aca-b978-a7a87e5aadfb None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:03.147098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025d5e6-3b94-4aca-b978-a7a87e5aadfb None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:03.147471 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025d5e6-3b94-4aca-b978-a7a87e5aadfb None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:03.148261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025d5e6-3b94-4aca-b978-a7a87e5aadfb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:03.148716 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025d5e6-3b94-4aca-b978-a7a87e5aadfb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:03.150282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b025d5e6-3b94-4aca-b978-a7a87e5aadfb None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:03.162807 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 546/1042] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:21:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:21:03.164362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3257a0-8ea5-450c-b0fe-cd3ba438bc15 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:03.164655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3257a0-8ea5-450c-b0fe-cd3ba438bc15 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:03.164953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3257a0-8ea5-450c-b0fe-cd3ba438bc15 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:03.170362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 547/1043] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:03.171210 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3257a0-8ea5-450c-b0fe-cd3ba438bc15 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:03.181339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29474a0c-514d-4dac-b437-2022ee7921fa None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:03.215850 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209a39e-682f-47e5-a6f1-36a10746af6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:03.216503 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209a39e-682f-47e5-a6f1-36a10746af6a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:03.216835 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209a39e-682f-47e5-a6f1-36a10746af6a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:03.217110 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209a39e-682f-47e5-a6f1-36a10746af6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:03.217838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b209a39e-682f-47e5-a6f1-36a10746af6a None admin] RBAC: Adding query filter params (interface=public, service_id=70d6326fe45044808962718cff9774eb, region_id=RegionOne) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:03.218145 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b209a39e-682f-47e5-a6f1-36a10746af6a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:03.219507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b209a39e-682f-47e5-a6f1-36a10746af6a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:03.224456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 497/1044] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Aug 29 20:21:03 2022] GET /identity/v3/endpoints?service_id=70d6326fe45044808962718cff9774eb&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:03.224456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8469504-1336-4170-ae95-1488d787dbf5 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:03.224456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8469504-1336-4170-ae95-1488d787dbf5 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:03.224456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8469504-1336-4170-ae95-1488d787dbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:03.243602 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-f8469504-1336-4170-ae95-1488d787dbf5 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:03.244821 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f8469504-1336-4170-ae95-1488d787dbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:03.274981 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3257a0-8ea5-450c-b0fe-cd3ba438bc15 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:03.314197 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ebe49a-5fa4-4957-a4ec-0536a92543a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:03.315035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebe49a-5fa4-4957-a4ec-0536a92543a4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:03.315358 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebe49a-5fa4-4957-a4ec-0536a92543a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:03.315726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebe49a-5fa4-4957-a4ec-0536a92543a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:03.316613 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ebe49a-5fa4-4957-a4ec-0536a92543a4 None admin] RBAC: Adding query filter params (role.id=8ece00b091b347179365b72919fdd7dc, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=c0c44754a1c04a2b87081aba6cb509d3) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:03.317002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ebe49a-5fa4-4957-a4ec-0536a92543a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:03.318587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ebe49a-5fa4-4957-a4ec-0536a92543a4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:03.319223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 498/1045] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:03 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:03.320096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8469504-1336-4170-ae95-1488d787dbf5 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:03.336257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 548/1046] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:21:03 2022] GET /identity/v3/role_assignments?user.id=c0c44754a1c04a2b87081aba6cb509d3&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=8ece00b091b347179365b72919fdd7dc => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:03.369286 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:03.450411 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8469504-1336-4170-ae95-1488d787dbf5 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:03.501114 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eef4da-cf2e-4702-a67d-e7d8addd3bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:03.501114 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eef4da-cf2e-4702-a67d-e7d8addd3bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:03.501114 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eef4da-cf2e-4702-a67d-e7d8addd3bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:03.503167 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eef4da-cf2e-4702-a67d-e7d8addd3bb5 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:03.503167 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eef4da-cf2e-4702-a67d-e7d8addd3bb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:03.503624 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eef4da-cf2e-4702-a67d-e7d8addd3bb5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:03.519968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 499/1047] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Aug 29 20:21:03 2022] GET /identity/v3/services => generated 1957 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:21:03.591185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ebe49a-5fa4-4957-a4ec-0536a92543a4 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:03.624516 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:03.699521 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ebe49a-5fa4-4957-a4ec-0536a92543a4 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:03.731967 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2193c-ffab-4b35-acbf-a07bb091b29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:03.732608 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2193c-ffab-4b35-acbf-a07bb091b29b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:03.732888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2193c-ffab-4b35-acbf-a07bb091b29b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:03.733160 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2193c-ffab-4b35-acbf-a07bb091b29b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:03.743413 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2193c-ffab-4b35-acbf-a07bb091b29b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:21:03.748788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-54a2193c-ffab-4b35-acbf-a07bb091b29b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:21:03.748788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:03.748788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:03.748788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:03.748788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:03.748788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:03.748788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:03.748788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:03.748788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:03.748788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:03.748788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:03.748788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:03.748788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:21:03.748788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:21:03.748788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:21:03.748788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:21:03.749784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:21:03.749784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:21:03.749784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:21:03.749784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:21:03.749784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:03.749784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:03.749784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:03.749784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:03.749784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:03.749784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:03.749784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:03.749784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:03.749784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:03.749784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:03.749784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:03.749784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:03.750628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:03.750628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:03.750628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:03.750628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:03.750628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:21:03.750628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:21:03.750628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:21:03.750628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:21:03.750628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:21:03.750628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:21:03.750628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:21:03.750628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:21:03.750628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 549/1048] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:21:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:21:03.755916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eef4da-cf2e-4702-a67d-e7d8addd3bb5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:03.859850 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:03.867033 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eef4da-cf2e-4702-a67d-e7d8addd3bb5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:03.903095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3cefa-6060-45db-a01e-b05b848f5b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:03.903610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3cefa-6060-45db-a01e-b05b848f5b5c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:03.903885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3cefa-6060-45db-a01e-b05b848f5b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:03.904195 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3cefa-6060-45db-a01e-b05b848f5b5c None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:03.904838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a3cefa-6060-45db-a01e-b05b848f5b5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:03.905139 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a3cefa-6060-45db-a01e-b05b848f5b5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:03.906489 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a3cefa-6060-45db-a01e-b05b848f5b5c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:03.920083 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 500/1049] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:21:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:21:03.921719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2193c-ffab-4b35-acbf-a07bb091b29b None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:04.030570 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:04.042368 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2193c-ffab-4b35-acbf-a07bb091b29b None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:04.076055 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1820c2-f53f-4a4e-a00f-eff99290e46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:04.076692 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1820c2-f53f-4a4e-a00f-eff99290e46d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:04.076970 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1820c2-f53f-4a4e-a00f-eff99290e46d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:04.077294 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1820c2-f53f-4a4e-a00f-eff99290e46d None admin] PATH_INFO: `/v3/users/a78ec1656f174f37936c781e0425485e` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:04.090765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1820c2-f53f-4a4e-a00f-eff99290e46d None admin] RBAC: Authorizing `identity:get_user(user_id=a78ec1656f174f37936c781e0425485e)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:04.092225 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1820c2-f53f-4a4e-a00f-eff99290e46d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:04.098487 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 550/1050] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:21:03 2022] GET /identity/v3/users/a78ec1656f174f37936c781e0425485e => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:04.100089 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3cefa-6060-45db-a01e-b05b848f5b5c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:04.206169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3cefa-6060-45db-a01e-b05b848f5b5c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:04.243132 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971c3ad-8802-4499-906a-bee9d5bea684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:04.243746 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971c3ad-8802-4499-906a-bee9d5bea684 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:04.244064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971c3ad-8802-4499-906a-bee9d5bea684 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:04.244900 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971c3ad-8802-4499-906a-bee9d5bea684 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:04.281852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:04.341104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971c3ad-8802-4499-906a-bee9d5bea684 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:04.342638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971c3ad-8802-4499-906a-bee9d5bea684 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:04.455611 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 501/1051] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:21:04 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:21:04.461261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1820c2-f53f-4a4e-a00f-eff99290e46d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:04.573315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1820c2-f53f-4a4e-a00f-eff99290e46d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:04.608172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16cd676-cb47-47e7-ba52-4c723dc1a27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:04.608803 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cd676-cb47-47e7-ba52-4c723dc1a27f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:04.609079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cd676-cb47-47e7-ba52-4c723dc1a27f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:04.609345 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cd676-cb47-47e7-ba52-4c723dc1a27f None admin] PATH_INFO: `/v3/projects/0c4a3bb2e1884623b3a8a3451b2c4c83` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:04.610423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16cd676-cb47-47e7-ba52-4c723dc1a27f None admin] RBAC: Authorizing `identity:get_project(project_id=0c4a3bb2e1884623b3a8a3451b2c4c83)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:04.612191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16cd676-cb47-47e7-ba52-4c723dc1a27f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:04.618842 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 551/1052] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:21:04 2022] GET /identity/v3/projects/0c4a3bb2e1884623b3a8a3451b2c4c83 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:04.620072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971c3ad-8802-4499-906a-bee9d5bea684 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:04.680985 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:04.730795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971c3ad-8802-4499-906a-bee9d5bea684 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:04.765214 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936528b-b4d8-4d27-a115-2b32b3f91b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:04.765828 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936528b-b4d8-4d27-a115-2b32b3f91b77 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:04.766065 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936528b-b4d8-4d27-a115-2b32b3f91b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:04.766415 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936528b-b4d8-4d27-a115-2b32b3f91b77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:04.767134 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936528b-b4d8-4d27-a115-2b32b3f91b77 None admin] RBAC: Adding query filter params (role.id=322da7bd64024770a64bdb73e5c54719, scope.project.id=0c4a3bb2e1884623b3a8a3451b2c4c83, user.id=a78ec1656f174f37936c781e0425485e) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:04.767433 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936528b-b4d8-4d27-a115-2b32b3f91b77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:04.768874 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936528b-b4d8-4d27-a115-2b32b3f91b77 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:04.779961 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 502/1053] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:21:04 2022] GET /identity/v3/role_assignments?user.id=a78ec1656f174f37936c781e0425485e&scope.project.id=0c4a3bb2e1884623b3a8a3451b2c4c83&role.id=322da7bd64024770a64bdb73e5c54719 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:05.274547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d3567-67c8-40d3-8665-5819f64b5349 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:05.275067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d3567-67c8-40d3-8665-5819f64b5349 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:05.275067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d3567-67c8-40d3-8665-5819f64b5349 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:05.276567 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 552/1054] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:05.287970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5acaa-9721-4b18-9c75-8b6e0b5dc1b9 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:05.287970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5acaa-9721-4b18-9c75-8b6e0b5dc1b9 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:05.287970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5acaa-9721-4b18-9c75-8b6e0b5dc1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:05.312864 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-beb5acaa-9721-4b18-9c75-8b6e0b5dc1b9 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:05.314580 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-beb5acaa-9721-4b18-9c75-8b6e0b5dc1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:05.401616 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 503/1055] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:05.501403 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfcad0b-e6bb-4cc0-8a0a-342638e1ed08 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:05.501724 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfcad0b-e6bb-4cc0-8a0a-342638e1ed08 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:05.501948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfcad0b-e6bb-4cc0-8a0a-342638e1ed08 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:05.503405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 553/1056] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:21:05.509024 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884536b3-78f2-4c42-aadb-325817bd3b15 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:05.509024 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884536b3-78f2-4c42-aadb-325817bd3b15 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:05.509024 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884536b3-78f2-4c42-aadb-325817bd3b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:05.527065 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-884536b3-78f2-4c42-aadb-325817bd3b15 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:05.528201 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-884536b3-78f2-4c42-aadb-325817bd3b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:05.576454 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:05.597187 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 504/1057] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:05.671792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfcad0b-e6bb-4cc0-8a0a-342638e1ed08 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:05.690608 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:05.783482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfcad0b-e6bb-4cc0-8a0a-342638e1ed08 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:05.826938 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed2438-0afb-4ed4-8985-275763571b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:05.828513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed2438-0afb-4ed4-8985-275763571b25 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:05.829190 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed2438-0afb-4ed4-8985-275763571b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:05.829697 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed2438-0afb-4ed4-8985-275763571b25 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:05.845788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ed2438-0afb-4ed4-8985-275763571b25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:21:05.850609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-f9ed2438-0afb-4ed4-8985-275763571b25 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:21:05.850609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:05.850609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:05.850609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:05.850609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:05.850609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:05.850609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:05.850609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:05.850609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:05.850609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:05.850609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:05.850609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:05.850609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:21:05.850609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:21:05.850609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:21:05.850609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:21:05.851542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:21:05.851542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:21:05.851542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:21:05.851542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:21:05.851542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:05.851542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:05.851542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:05.851542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:05.851542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:05.851542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:05.851542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:05.851542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:05.851542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:05.851542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:05.851542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:05.851542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:05.857058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:05.857058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:05.857058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:05.857058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:05.857058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:21:05.857058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:21:05.857058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:21:05.857058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:21:05.857058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:21:05.857058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:21:05.857058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:21:05.857058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:21:05.857058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 554/1058] 10.222.0.18 () {56 vars in 1120 bytes} [Mon Aug 29 20:21:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:21:05.858640 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884536b3-78f2-4c42-aadb-325817bd3b15 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:05.861883 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed2438-0afb-4ed4-8985-275763571b25 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:05.951631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:05.975824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed2438-0afb-4ed4-8985-275763571b25 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:05.977488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884536b3-78f2-4c42-aadb-325817bd3b15 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:06.013700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff254d6-059b-4310-ad1a-0b067423ac07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:06.014499 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff254d6-059b-4310-ad1a-0b067423ac07 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:06.014880 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff254d6-059b-4310-ad1a-0b067423ac07 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:06.015224 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff254d6-059b-4310-ad1a-0b067423ac07 None admin] PATH_INFO: `/v3/services/image` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:06.015867 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff254d6-059b-4310-ad1a-0b067423ac07 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:06.017319 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff254d6-059b-4310-ad1a-0b067423ac07 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:06.022503 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e8f78-e1af-4697-ad20-3e95c410bd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:06.023113 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e8f78-e1af-4697-ad20-3e95c410bd24 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:06.023393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e8f78-e1af-4697-ad20-3e95c410bd24 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:06.023661 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e8f78-e1af-4697-ad20-3e95c410bd24 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:06.024349 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e8f78-e1af-4697-ad20-3e95c410bd24 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:06.024650 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e8f78-e1af-4697-ad20-3e95c410bd24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:06.026004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-8ff254d6-059b-4310-ad1a-0b067423ac07 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:21:06.026004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:06.026004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:21:06.026004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:21:06.026004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:21:06.026004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:21:06.026004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:21:06.026004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:21:06.026004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:21:06.026004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:21:06.026004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:21:06.026004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:21:06.026004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:06.026004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:06.026004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:06.026004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:06.026004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:06.026958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:06.026958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:06.026958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:06.026958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:06.026958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:06.026958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:06.026958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:21:06.026958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:21:06.026958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:21:06.026958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:21:06.026958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:06.026958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:06.026958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:06.026958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:06.026958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:06.026958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:06.027799 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:06.027799 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:06.027799 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:06.027799 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:06.027799 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:06.027799 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:06.027799 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:06.027799 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:06.027799 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:06.027799 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:06.027799 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:21:06.027799 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:21:06.027799 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 29 20:21:06.027799 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:21:06.029180 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e8f78-e1af-4697-ad20-3e95c410bd24 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:06.032055 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 555/1059] 10.222.0.18 () {56 vars in 1105 bytes} [Mon Aug 29 20:21:05 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:21:06.032891 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff254d6-059b-4310-ad1a-0b067423ac07 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:06.040632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 505/1060] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Aug 29 20:21:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:06.042216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e8f78-e1af-4697-ad20-3e95c410bd24 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:06.095567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:06.141044 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff254d6-059b-4310-ad1a-0b067423ac07 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:06.157891 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e8f78-e1af-4697-ad20-3e95c410bd24 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:06.188364 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad367fe9-5852-4a77-8f57-6bba5d810f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:06.188976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad367fe9-5852-4a77-8f57-6bba5d810f83 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:06.189261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad367fe9-5852-4a77-8f57-6bba5d810f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:06.189536 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad367fe9-5852-4a77-8f57-6bba5d810f83 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:06.190204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad367fe9-5852-4a77-8f57-6bba5d810f83 None admin] RBAC: Adding query filter params (name=image) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:06.190509 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad367fe9-5852-4a77-8f57-6bba5d810f83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:06.191881 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad367fe9-5852-4a77-8f57-6bba5d810f83 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:06.197512 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 556/1061] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:21:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:06.203201 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad367fe9-5852-4a77-8f57-6bba5d810f83 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:06.205438 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f4bf9-8bbc-4a94-a12b-097dac0e7a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:06.206195 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f4bf9-8bbc-4a94-a12b-097dac0e7a2b None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:06.206543 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f4bf9-8bbc-4a94-a12b-097dac0e7a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:06.206907 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f4bf9-8bbc-4a94-a12b-097dac0e7a2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:06.211435 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:06.219350 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f4bf9-8bbc-4a94-a12b-097dac0e7a2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:06.220836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f4bf9-8bbc-4a94-a12b-097dac0e7a2b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:06.236246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-df6f4bf9-8bbc-4a94-a12b-097dac0e7a2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:21:06.236246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:06.236246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:06.236246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:06.236246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:06.236246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:06.236246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:06.236246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:06.236246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:06.236246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:06.236246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:06.236246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:06.236246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:21:06.236246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:21:06.236246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:21:06.236246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:21:06.237266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:06.237266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:06.237266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:06.237266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:06.237266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:06.237266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:06.237266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:06.237266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:06.237266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:06.237266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:06.237266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:06.237266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:06.237266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:06.237266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:06.237266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:06.237266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:06.237266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:21:06.243723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:21:06.243723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:21:06.243723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:21:06.243723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 506/1062] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:21:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:21:06.243723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f4bf9-8bbc-4a94-a12b-097dac0e7a2b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:06.342606 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad367fe9-5852-4a77-8f57-6bba5d810f83 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:06.364154 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:06.373862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f4bf9-8bbc-4a94-a12b-097dac0e7a2b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:06.387416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65129f08-0a6f-4da4-96d3-e17189056472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:06.388864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65129f08-0a6f-4da4-96d3-e17189056472 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:06.389214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65129f08-0a6f-4da4-96d3-e17189056472 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:06.389497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65129f08-0a6f-4da4-96d3-e17189056472 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:06.390141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65129f08-0a6f-4da4-96d3-e17189056472 None admin] RBAC: Adding query filter params (type=image) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:06.390448 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65129f08-0a6f-4da4-96d3-e17189056472 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:06.391820 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65129f08-0a6f-4da4-96d3-e17189056472 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:06.402047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 557/1063] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:21:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:21:06.403150 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65129f08-0a6f-4da4-96d3-e17189056472 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:06.416353 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e12a0f-e640-45b0-9585-c086a1be60c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:06.417080 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e12a0f-e640-45b0-9585-c086a1be60c3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:06.417464 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e12a0f-e640-45b0-9585-c086a1be60c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:06.417828 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e12a0f-e640-45b0-9585-c086a1be60c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:06.418576 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e12a0f-e640-45b0-9585-c086a1be60c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:06.418994 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e12a0f-e640-45b0-9585-c086a1be60c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:06.420489 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e12a0f-e640-45b0-9585-c086a1be60c3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:06.434906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 507/1064] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:06.440686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e12a0f-e640-45b0-9585-c086a1be60c3 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:06.480565 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:06.517422 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65129f08-0a6f-4da4-96d3-e17189056472 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:06.551540 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e12a0f-e640-45b0-9585-c086a1be60c3 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:06.555694 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9991c-7903-43f4-8b27-18c0f29fea85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:06.557021 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9991c-7903-43f4-8b27-18c0f29fea85 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:06.557524 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9991c-7903-43f4-8b27-18c0f29fea85 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:06.557930 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9991c-7903-43f4-8b27-18c0f29fea85 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:06.558727 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9991c-7903-43f4-8b27-18c0f29fea85 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:06.560345 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9991c-7903-43f4-8b27-18c0f29fea85 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:06.584775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 558/1065] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:21:06 2022] POST /identity/v3/endpoints => generated 327 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:06.603167 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafbd35-a597-4658-b087-2ad63d616670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:06.603946 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafbd35-a597-4658-b087-2ad63d616670 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:06.604243 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafbd35-a597-4658-b087-2ad63d616670 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:06.604529 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafbd35-a597-4658-b087-2ad63d616670 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:06.614441 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dafbd35-a597-4658-b087-2ad63d616670 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:06.616065 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dafbd35-a597-4658-b087-2ad63d616670 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:06.630264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-9dafbd35-a597-4658-b087-2ad63d616670 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:21:06.630264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:06.630264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:06.630264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:06.630264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:06.630264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:06.630264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:06.630264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:06.630264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:06.630264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:06.630264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:06.630264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:06.630264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:21:06.630264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:21:06.630264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:21:06.630264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:21:06.631325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:06.631325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:06.631325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:21:06.631325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:21:06.631325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:21:06.631325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:21:06.631325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:06.631325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:06.631325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:06.631325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:06.631325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:06.631325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:06.631325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:06.631325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:06.631325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:06.631325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:06.631325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:06.631325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:06.632295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:06.632295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:06.632295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:21:06.632295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:21:06.632295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:21:06.632295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:21:06.632295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:21:06.632295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:21:06.632295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:21:06.632295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:21:06.632295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 508/1066] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Aug 29 20:21:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:21:06.638062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9991c-7903-43f4-8b27-18c0f29fea85 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:06.735554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eae489-eacf-4e5e-9c6a-5c84dc5da986 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:06.735834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eae489-eacf-4e5e-9c6a-5c84dc5da986 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:06.736140 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eae489-eacf-4e5e-9c6a-5c84dc5da986 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:06.737496 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 509/1067] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:06.743514 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b8dc4-1d7a-44f5-82cd-f2999e190839 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:06.744815 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b8dc4-1d7a-44f5-82cd-f2999e190839 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:06.744815 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b8dc4-1d7a-44f5-82cd-f2999e190839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:06.765874 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e54b8dc4-1d7a-44f5-82cd-f2999e190839 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:06.767553 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e54b8dc4-1d7a-44f5-82cd-f2999e190839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:06.771646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9991c-7903-43f4-8b27-18c0f29fea85 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:06.777684 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:06.802079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:06.816772 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff74ee-c129-4491-adc3-98746f4a8515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:06.816772 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff74ee-c129-4491-adc3-98746f4a8515 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:06.816772 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff74ee-c129-4491-adc3-98746f4a8515 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:06.817950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff74ee-c129-4491-adc3-98746f4a8515 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:06.818176 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff74ee-c129-4491-adc3-98746f4a8515 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:06.818559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff74ee-c129-4491-adc3-98746f4a8515 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:06.820088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff74ee-c129-4491-adc3-98746f4a8515 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:06.840198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 559/1068] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Aug 29 20:21:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:06.845705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff74ee-c129-4491-adc3-98746f4a8515 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:06.859975 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 510/1069] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:21:06.963376 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff74ee-c129-4491-adc3-98746f4a8515 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:07.004771 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41095912-e7a2-4954-adf1-9d4b4519bc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:07.005967 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41095912-e7a2-4954-adf1-9d4b4519bc1c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:07.006241 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41095912-e7a2-4954-adf1-9d4b4519bc1c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:07.006650 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41095912-e7a2-4954-adf1-9d4b4519bc1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:07.018000 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41095912-e7a2-4954-adf1-9d4b4519bc1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:07.019533 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41095912-e7a2-4954-adf1-9d4b4519bc1c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:07.034750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-41095912-e7a2-4954-adf1-9d4b4519bc1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:21:07.034750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:07.034750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:07.034750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:07.034750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:07.034750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:07.034750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:07.034750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:07.034750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:07.034750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:07.034750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:07.034750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:07.034750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:21:07.034750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:21:07.034750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:21:07.034750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:21:07.036088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:07.036088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:07.036088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:07.036088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:07.036088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:07.036088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:07.036088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:07.036088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:07.036088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:07.036088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:07.036088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:07.036088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:07.036088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:07.036088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:07.036088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:07.036088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:07.036088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:21:07.037212 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:21:07.037212 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:21:07.037212 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:21:07.037589 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 560/1070] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:21:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:21:07.045425 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b8dc4-1d7a-44f5-82cd-f2999e190839 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:07.146716 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41095912-e7a2-4954-adf1-9d4b4519bc1c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:07.168171 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b8dc4-1d7a-44f5-82cd-f2999e190839 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:07.185261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:07.204965 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94926a15-eeff-48d6-b2d3-e0f4dfb5ad14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:07.205727 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94926a15-eeff-48d6-b2d3-e0f4dfb5ad14 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:07.206128 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94926a15-eeff-48d6-b2d3-e0f4dfb5ad14 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:07.206573 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94926a15-eeff-48d6-b2d3-e0f4dfb5ad14 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:07.207266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94926a15-eeff-48d6-b2d3-e0f4dfb5ad14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:07.207667 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94926a15-eeff-48d6-b2d3-e0f4dfb5ad14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:07.209169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94926a15-eeff-48d6-b2d3-e0f4dfb5ad14 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:07.220715 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 511/1071] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:07.226180 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94926a15-eeff-48d6-b2d3-e0f4dfb5ad14 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:07.229803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:07.265114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41095912-e7a2-4954-adf1-9d4b4519bc1c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:07.309569 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa68c9-b359-4f15-9a11-16c44fa30268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:07.310212 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa68c9-b359-4f15-9a11-16c44fa30268 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:07.310499 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa68c9-b359-4f15-9a11-16c44fa30268 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:07.310780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa68c9-b359-4f15-9a11-16c44fa30268 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:07.322252 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfa68c9-b359-4f15-9a11-16c44fa30268 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:21:07.327814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-2cfa68c9-b359-4f15-9a11-16c44fa30268 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:21:07.327814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:07.327814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:07.327814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:07.327814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:07.327814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:07.327814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:07.327814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:07.327814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:07.327814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:07.327814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:07.327814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:07.327814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:21:07.327814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:21:07.327814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:21:07.327814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:21:07.329391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:21:07.329391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:21:07.329391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:21:07.329391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:21:07.329391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:07.329391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:07.329391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:07.329391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:07.329391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:07.329391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:07.329391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:07.329391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:07.329391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:07.329391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:07.329391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:07.329391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:07.331153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:07.331153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:07.331153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:07.331153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:07.331153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:21:07.331153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:21:07.331153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:21:07.331153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:21:07.331153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:21:07.331153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:21:07.331153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:21:07.331153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:21:07.331153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 561/1072] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:21:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:21:07.335168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa68c9-b359-4f15-9a11-16c44fa30268 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:07.354633 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94926a15-eeff-48d6-b2d3-e0f4dfb5ad14 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:07.404931 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2198bd5-e740-4b20-a675-6c3bf4b1fa00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:07.406059 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2198bd5-e740-4b20-a675-6c3bf4b1fa00 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:07.406349 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2198bd5-e740-4b20-a675-6c3bf4b1fa00 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:07.406899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2198bd5-e740-4b20-a675-6c3bf4b1fa00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:07.423639 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2198bd5-e740-4b20-a675-6c3bf4b1fa00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:07.425159 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2198bd5-e740-4b20-a675-6c3bf4b1fa00 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:07.437592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-c2198bd5-e740-4b20-a675-6c3bf4b1fa00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:21:07.437592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:07.437592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:07.437592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:07.437592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:07.437592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:07.437592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:07.437592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:07.437592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:07.437592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:07.437592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:07.437592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:07.437592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:21:07.437592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:21:07.437592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:21:07.438762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:21:07.438762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:07.438762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:07.438762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:07.438762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:07.438762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:07.438762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:07.438762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:07.438762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:07.438762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:07.438762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:07.438762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:07.438762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:07.438762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:07.438762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:07.438762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:07.438762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:07.439755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:21:07.439755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:21:07.439755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:21:07.439755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:21:07.439755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:21:07.439755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:21:07.439755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:21:07.439755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:21:07.439755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 512/1073] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:21:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:21:07.444103 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:07.444564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2198bd5-e740-4b20-a675-6c3bf4b1fa00 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:07.452481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa68c9-b359-4f15-9a11-16c44fa30268 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:07.493642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88ea71-4b86-4ab1-b36c-e93b5930fec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:07.494134 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88ea71-4b86-4ab1-b36c-e93b5930fec1 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:07.494413 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88ea71-4b86-4ab1-b36c-e93b5930fec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:07.494688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88ea71-4b86-4ab1-b36c-e93b5930fec1 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:07.495349 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf88ea71-4b86-4ab1-b36c-e93b5930fec1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:07.495650 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf88ea71-4b86-4ab1-b36c-e93b5930fec1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:07.497062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf88ea71-4b86-4ab1-b36c-e93b5930fec1 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:07.506556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 562/1074] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:21:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:21:07.512186 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88ea71-4b86-4ab1-b36c-e93b5930fec1 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:07.518054 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:07.564341 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2198bd5-e740-4b20-a675-6c3bf4b1fa00 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:07.600330 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3635e-9c08-4e20-b18c-5702fc7279cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:07.600892 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3635e-9c08-4e20-b18c-5702fc7279cb None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:07.601325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3635e-9c08-4e20-b18c-5702fc7279cb None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:07.601625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca3635e-9c08-4e20-b18c-5702fc7279cb None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:07.602377 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3635e-9c08-4e20-b18c-5702fc7279cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:07.602686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3635e-9c08-4e20-b18c-5702fc7279cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:07.607977 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca3635e-9c08-4e20-b18c-5702fc7279cb None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:07.622331 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 513/1075] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:21:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:21:07.623428 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3635e-9c08-4e20-b18c-5702fc7279cb None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:07.629739 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88ea71-4b86-4ab1-b36c-e93b5930fec1 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:07.668600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fefea7-186b-45df-91cb-ad667baaea1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:07.669155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fefea7-186b-45df-91cb-ad667baaea1d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:07.669431 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fefea7-186b-45df-91cb-ad667baaea1d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:07.669707 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fefea7-186b-45df-91cb-ad667baaea1d None admin] PATH_INFO: `/v3/users/a78ec1656f174f37936c781e0425485e` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:07.670831 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fefea7-186b-45df-91cb-ad667baaea1d None admin] RBAC: Authorizing `identity:get_user(user_id=a78ec1656f174f37936c781e0425485e)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:07.672278 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fefea7-186b-45df-91cb-ad667baaea1d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:07.674027 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 563/1076] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:21:07 2022] GET /identity/v3/users/a78ec1656f174f37936c781e0425485e => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:21:07.679529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fefea7-186b-45df-91cb-ad667baaea1d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:07.706992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:07.734472 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca3635e-9c08-4e20-b18c-5702fc7279cb None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:07.759395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:07.774212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9af5d-e2fb-4b43-80c9-95635dc0c94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:07.774872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9af5d-e2fb-4b43-80c9-95635dc0c94b None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:07.775149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9af5d-e2fb-4b43-80c9-95635dc0c94b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:07.776028 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9af5d-e2fb-4b43-80c9-95635dc0c94b None admin] PATH_INFO: `/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/c0c44754a1c04a2b87081aba6cb509d3/roles/8ece00b091b347179365b72919fdd7dc` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:07.778258 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9af5d-e2fb-4b43-80c9-95635dc0c94b None admin] RBAC: Authorizing `identity:create_grant(project_id=90b25dc95d5a4a068994bdd801d6b8ea, user_id=c0c44754a1c04a2b87081aba6cb509d3, role_id=8ece00b091b347179365b72919fdd7dc)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:07.779686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9af5d-e2fb-4b43-80c9-95635dc0c94b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:07.793763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 514/1077] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:21:07 2022] PUT /identity/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/c0c44754a1c04a2b87081aba6cb509d3/roles/8ece00b091b347179365b72919fdd7dc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:21:07.811377 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fefea7-186b-45df-91cb-ad667baaea1d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:07.852883 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537d4f0-d66a-4a84-8e81-18f7e881b7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:07.853608 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537d4f0-d66a-4a84-8e81-18f7e881b7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:07.853884 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537d4f0-d66a-4a84-8e81-18f7e881b7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:07.854148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537d4f0-d66a-4a84-8e81-18f7e881b7c2 None admin] PATH_INFO: `/v3/projects/0c4a3bb2e1884623b3a8a3451b2c4c83` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:07.855321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537d4f0-d66a-4a84-8e81-18f7e881b7c2 None admin] RBAC: Authorizing `identity:get_project(project_id=0c4a3bb2e1884623b3a8a3451b2c4c83)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:07.856807 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537d4f0-d66a-4a84-8e81-18f7e881b7c2 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:07.858728 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 564/1078] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:21:07 2022] GET /identity/v3/projects/0c4a3bb2e1884623b3a8a3451b2c4c83 => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:21:07.864923 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9af5d-e2fb-4b43-80c9-95635dc0c94b None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:07.981978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9af5d-e2fb-4b43-80c9-95635dc0c94b None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:08.019282 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8decbad-a3f6-45b2-bbd3-e6c0088dde87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:08.019962 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8decbad-a3f6-45b2-bbd3-e6c0088dde87 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:08.020244 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8decbad-a3f6-45b2-bbd3-e6c0088dde87 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:08.020523 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8decbad-a3f6-45b2-bbd3-e6c0088dde87 None admin] PATH_INFO: `/v3/projects/0c4a3bb2e1884623b3a8a3451b2c4c83/users/a78ec1656f174f37936c781e0425485e/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:08.022450 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8decbad-a3f6-45b2-bbd3-e6c0088dde87 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c4a3bb2e1884623b3a8a3451b2c4c83, user_id=a78ec1656f174f37936c781e0425485e, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:08.023878 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8decbad-a3f6-45b2-bbd3-e6c0088dde87 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:08.037829 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 515/1079] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:21:07 2022] PUT /identity/v3/projects/0c4a3bb2e1884623b3a8a3451b2c4c83/users/a78ec1656f174f37936c781e0425485e/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:08.597251 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578361f1-05aa-4b58-bf69-08032ed6ea23 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:08.597768 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578361f1-05aa-4b58-bf69-08032ed6ea23 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:08.597768 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578361f1-05aa-4b58-bf69-08032ed6ea23 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:08.599100 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 565/1080] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:08.605091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cb184-2504-47ff-b699-f09401812cb1 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:08.605381 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cb184-2504-47ff-b699-f09401812cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:08.605657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cb184-2504-47ff-b699-f09401812cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:08.625329 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e87cb184-2504-47ff-b699-f09401812cb1 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:08.626536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e87cb184-2504-47ff-b699-f09401812cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:08.669587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:08.708311 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 516/1081] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:08.987366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578361f1-05aa-4b58-bf69-08032ed6ea23 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:09.094135 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:09.106874 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578361f1-05aa-4b58-bf69-08032ed6ea23 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:09.147416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb5288-16d5-432d-b9fa-fe8fa31faa11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:09.148174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb5288-16d5-432d-b9fa-fe8fa31faa11 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:09.148461 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb5288-16d5-432d-b9fa-fe8fa31faa11 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:09.148732 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb5288-16d5-432d-b9fa-fe8fa31faa11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:09.226670 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb5288-16d5-432d-b9fa-fe8fa31faa11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:09.228874 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb5288-16d5-432d-b9fa-fe8fa31faa11 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:09.342894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:09.359999 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 566/1082] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:21:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 20:21:09.361529 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cb184-2504-47ff-b699-f09401812cb1 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:09.466637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cb184-2504-47ff-b699-f09401812cb1 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:09.504035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f569d4d-0fba-4595-8dc8-4e95eb21a433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:09.504678 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f569d4d-0fba-4595-8dc8-4e95eb21a433 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:09.505007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f569d4d-0fba-4595-8dc8-4e95eb21a433 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:09.505319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f569d4d-0fba-4595-8dc8-4e95eb21a433 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:09.516689 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f569d4d-0fba-4595-8dc8-4e95eb21a433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:09.518299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f569d4d-0fba-4595-8dc8-4e95eb21a433 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:09.537226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-4f569d4d-0fba-4595-8dc8-4e95eb21a433 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:21:09.537226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:09.537226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:09.537226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:09.537226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:09.537226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:09.537226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:09.537226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:09.537226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:09.537226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:09.537226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:09.537226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:09.537226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:21:09.537226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:21:09.537226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:21:09.537226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:21:09.538517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:09.538517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:09.538517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:09.538517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:09.538517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:09.538517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:09.538517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:09.538517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:09.538517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:09.538517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:09.538517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:09.538517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:09.538517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:09.538517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:09.538517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:09.538517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:09.538517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:21:09.546012 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:21:09.546012 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:21:09.546012 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:21:09.546012 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 517/1083] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:21:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:21:09.549717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb5288-16d5-432d-b9fa-fe8fa31faa11 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:09.660892 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb5288-16d5-432d-b9fa-fe8fa31faa11 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:09.697118 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb822c7b-1dc9-4b45-bda2-3c1fa8b700da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:09.697741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb822c7b-1dc9-4b45-bda2-3c1fa8b700da None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:09.698113 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb822c7b-1dc9-4b45-bda2-3c1fa8b700da None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:09.698413 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb822c7b-1dc9-4b45-bda2-3c1fa8b700da None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:09.699067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb822c7b-1dc9-4b45-bda2-3c1fa8b700da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:09.699372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb822c7b-1dc9-4b45-bda2-3c1fa8b700da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:09.700762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb822c7b-1dc9-4b45-bda2-3c1fa8b700da None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:09.716371 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 567/1084] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:09.751963 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f3861-4445-40a7-873b-9c0117165c11 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:09.752284 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f3861-4445-40a7-873b-9c0117165c11 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:09.752560 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f3861-4445-40a7-873b-9c0117165c11 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:09.753974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 518/1085] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:09.760491 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1ad9b-5546-4fb8-8269-4212f1dd1bd4 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:09.760936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1ad9b-5546-4fb8-8269-4212f1dd1bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:09.761172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1ad9b-5546-4fb8-8269-4212f1dd1bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:09.780860 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-56f1ad9b-5546-4fb8-8269-4212f1dd1bd4 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:09.782098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-56f1ad9b-5546-4fb8-8269-4212f1dd1bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:09.843507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:09.867967 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 568/1086] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:21:10.051151 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7c542-abba-4673-812e-d2d9a6ca401f None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:10.051431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7c542-abba-4673-812e-d2d9a6ca401f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:10.051707 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7c542-abba-4673-812e-d2d9a6ca401f None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:10.053124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 519/1087] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:21:10.059327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b5cb6-50dc-4806-b0f9-c53275648838 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:10.059614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b5cb6-50dc-4806-b0f9-c53275648838 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:10.059907 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b5cb6-50dc-4806-b0f9-c53275648838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:10.078249 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-8f7b5cb6-50dc-4806-b0f9-c53275648838 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:10.079438 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-8f7b5cb6-50dc-4806-b0f9-c53275648838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:10.136012 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7c542-abba-4673-812e-d2d9a6ca401f None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:10.148415 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 569/1088] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:10.161834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:10.241212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7c542-abba-4673-812e-d2d9a6ca401f None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:10.274622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f3ef9-2415-4963-9b66-b40a024ae713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:10.275785 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f3ef9-2415-4963-9b66-b40a024ae713 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:10.276094 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f3ef9-2415-4963-9b66-b40a024ae713 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:10.276368 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f3ef9-2415-4963-9b66-b40a024ae713 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:10.287189 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f3ef9-2415-4963-9b66-b40a024ae713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:21:10.292921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-037f3ef9-2415-4963-9b66-b40a024ae713 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:21:10.292921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:10.292921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:10.292921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:10.292921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:10.292921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:10.292921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:10.292921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:10.292921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:10.292921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:10.292921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:10.292921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:10.292921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:21:10.292921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:21:10.292921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:21:10.292921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:21:10.293938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:21:10.293938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:21:10.293938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:21:10.293938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:21:10.293938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:10.293938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:10.293938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:10.293938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:10.293938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:10.293938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:10.293938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:10.293938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:10.293938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:10.293938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:10.293938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:10.293938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:10.294763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:10.294763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:10.294763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:10.294763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:10.294763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:21:10.294763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:21:10.294763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:21:10.294763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:21:10.294763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:21:10.294763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:21:10.294763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 29 20:21:10.294763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:21:10.296293 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 520/1089] 10.222.0.18 () {56 vars in 1120 bytes} [Mon Aug 29 20:21:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:21:10.300184 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b5cb6-50dc-4806-b0f9-c53275648838 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:10.401936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b5cb6-50dc-4806-b0f9-c53275648838 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:10.415455 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:10.425280 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f3ef9-2415-4963-9b66-b40a024ae713 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:10.439711 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c606ab5-367a-40d6-abab-d22ac1129461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:10.440476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c606ab5-367a-40d6-abab-d22ac1129461 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:10.440947 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c606ab5-367a-40d6-abab-d22ac1129461 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:10.441330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c606ab5-367a-40d6-abab-d22ac1129461 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:10.442316 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c606ab5-367a-40d6-abab-d22ac1129461 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:10.442713 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c606ab5-367a-40d6-abab-d22ac1129461 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:10.444588 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c606ab5-367a-40d6-abab-d22ac1129461 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:10.454926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 570/1090] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Aug 29 20:21:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:10.461256 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c606ab5-367a-40d6-abab-d22ac1129461 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:10.521399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:10.537655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f3ef9-2415-4963-9b66-b40a024ae713 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:10.569742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c606ab5-367a-40d6-abab-d22ac1129461 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:10.584321 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473d4d9-8b7d-4ded-8213-2c8bba7bfe81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:10.584963 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473d4d9-8b7d-4ded-8213-2c8bba7bfe81 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:10.585242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473d4d9-8b7d-4ded-8213-2c8bba7bfe81 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:10.585514 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473d4d9-8b7d-4ded-8213-2c8bba7bfe81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:10.596611 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473d4d9-8b7d-4ded-8213-2c8bba7bfe81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:21:10.601440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-8473d4d9-8b7d-4ded-8213-2c8bba7bfe81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:21:10.601440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:10.601440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:10.601440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:10.601440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:10.601440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:10.601440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:10.601440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:10.601440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:10.601440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:10.601440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:10.601440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:10.601440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:21:10.601440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:21:10.601440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:21:10.601440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:21:10.602554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:21:10.602554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:21:10.602554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:21:10.602554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:21:10.602554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:10.602554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:10.602554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:10.602554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:10.602554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:10.602554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:10.602554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:10.602554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:10.602554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:10.602554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:10.602554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:10.602554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:10.604693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:10.604693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:10.604693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:10.604693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:10.604693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:21:10.604693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:21:10.604693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:21:10.604693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:21:10.604693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:21:10.604693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:21:10.604693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:21:10.604693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:21:10.604693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 521/1091] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:21:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:21:10.610119 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a528f-fd0c-4cd2-8225-9ccf9489d1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:10.610790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a528f-fd0c-4cd2-8225-9ccf9489d1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:10.611074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a528f-fd0c-4cd2-8225-9ccf9489d1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:10.611352 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a528f-fd0c-4cd2-8225-9ccf9489d1f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:10.616024 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473d4d9-8b7d-4ded-8213-2c8bba7bfe81 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:10.626571 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a528f-fd0c-4cd2-8225-9ccf9489d1f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:10.628061 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a528f-fd0c-4cd2-8225-9ccf9489d1f7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:10.643858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-8b6a528f-fd0c-4cd2-8225-9ccf9489d1f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:21:10.643858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:10.643858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:10.643858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:10.643858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:10.643858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:10.643858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:10.643858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:10.643858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:10.643858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:10.643858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:10.643858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:10.643858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:21:10.643858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:21:10.643858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:21:10.643858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:21:10.645074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:10.645074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:10.645074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:10.645074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:10.645074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:10.645074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:10.645074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:10.645074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:10.645074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:10.645074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:10.645074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:10.645074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:10.645074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:10.645074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:10.645074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:10.645074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:10.645074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:21:10.646343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:21:10.646343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:21:10.646343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:21:10.646343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 571/1092] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:21:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:21:10.651887 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a528f-fd0c-4cd2-8225-9ccf9489d1f7 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:10.679498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:10.731356 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473d4d9-8b7d-4ded-8213-2c8bba7bfe81 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:10.755244 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a528f-fd0c-4cd2-8225-9ccf9489d1f7 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:10.767903 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bffbd-4eb4-44f0-be6a-c6e6cc88899d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:10.768546 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bffbd-4eb4-44f0-be6a-c6e6cc88899d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:10.768829 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bffbd-4eb4-44f0-be6a-c6e6cc88899d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:10.769108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bffbd-4eb4-44f0-be6a-c6e6cc88899d None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:10.769776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1bffbd-4eb4-44f0-be6a-c6e6cc88899d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:10.770090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1bffbd-4eb4-44f0-be6a-c6e6cc88899d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:10.771458 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1bffbd-4eb4-44f0-be6a-c6e6cc88899d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:10.781121 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 522/1093] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:21:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:10.787042 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bffbd-4eb4-44f0-be6a-c6e6cc88899d None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:10.794860 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ba667-297d-4b64-8e71-6ce2f23ff800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:10.795474 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ba667-297d-4b64-8e71-6ce2f23ff800 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:10.795753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ba667-297d-4b64-8e71-6ce2f23ff800 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:10.796056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ba667-297d-4b64-8e71-6ce2f23ff800 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:10.796521 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:10.796696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ba667-297d-4b64-8e71-6ce2f23ff800 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:10.797004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ba667-297d-4b64-8e71-6ce2f23ff800 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:10.798474 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ba667-297d-4b64-8e71-6ce2f23ff800 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:10.815309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 572/1094] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:21:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:21:10.816617 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ba667-297d-4b64-8e71-6ce2f23ff800 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:10.893403 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bffbd-4eb4-44f0-be6a-c6e6cc88899d None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:10.903051 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:10.918954 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ba667-297d-4b64-8e71-6ce2f23ff800 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:10.927619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dd95f-9e27-46e5-a55c-947409b57878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:10.928290 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dd95f-9e27-46e5-a55c-947409b57878 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:10.928575 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dd95f-9e27-46e5-a55c-947409b57878 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:10.928845 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dd95f-9e27-46e5-a55c-947409b57878 None admin] PATH_INFO: `/v3/users/a78ec1656f174f37936c781e0425485e` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:10.929989 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860dd95f-9e27-46e5-a55c-947409b57878 None admin] RBAC: Authorizing `identity:get_user(user_id=a78ec1656f174f37936c781e0425485e)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:10.931448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860dd95f-9e27-46e5-a55c-947409b57878 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:10.933270 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 523/1095] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:21:10 2022] GET /identity/v3/users/a78ec1656f174f37936c781e0425485e => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:21:10.939078 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dd95f-9e27-46e5-a55c-947409b57878 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:10.958610 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e940f2e-41e1-4726-a581-848443e13cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:10.959248 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e940f2e-41e1-4726-a581-848443e13cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:10.959518 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e940f2e-41e1-4726-a581-848443e13cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:10.959792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e940f2e-41e1-4726-a581-848443e13cc0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:10.967638 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e940f2e-41e1-4726-a581-848443e13cc0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:10.969153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e940f2e-41e1-4726-a581-848443e13cc0 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:10.980753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-3e940f2e-41e1-4726-a581-848443e13cc0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:21:10.980753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:10.980753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:10.980753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:10.980753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:10.980753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:10.980753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:10.980753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:10.980753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:10.980753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:10.980753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:10.980753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:10.980753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:21:10.980753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:21:10.980753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:21:10.980753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:21:10.982002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:10.982002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:10.982002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:21:10.982002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:21:10.982002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:21:10.982002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:21:10.982002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:10.982002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:10.982002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:10.982002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:10.982002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:10.982002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:10.982002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:10.982002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:10.982002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:10.982002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:10.982002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:10.982002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:10.982973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:10.982973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:10.982973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:21:10.982973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:21:10.982973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:21:10.982973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:21:10.982973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:21:10.982973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:21:10.982973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 29 20:21:10.982973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:21:10.982973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 573/1096] 10.222.0.18 () {56 vars in 1093 bytes} [Mon Aug 29 20:21:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:21:10.987574 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e940f2e-41e1-4726-a581-848443e13cc0 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:11.017882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:11.045762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dd95f-9e27-46e5-a55c-947409b57878 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:11.081313 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dbd91e-e0d2-4a91-8d28-48fedd02df42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:11.081956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbd91e-e0d2-4a91-8d28-48fedd02df42 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:11.082242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbd91e-e0d2-4a91-8d28-48fedd02df42 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:11.082520 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbd91e-e0d2-4a91-8d28-48fedd02df42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:11.093732 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e940f2e-41e1-4726-a581-848443e13cc0 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:11.128486 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51016e71-78be-40be-861f-c27cee2c6097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:11.129127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51016e71-78be-40be-861f-c27cee2c6097 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:11.129412 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51016e71-78be-40be-861f-c27cee2c6097 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:11.129689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51016e71-78be-40be-861f-c27cee2c6097 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:11.130479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51016e71-78be-40be-861f-c27cee2c6097 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:11.130776 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51016e71-78be-40be-861f-c27cee2c6097 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:11.132264 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51016e71-78be-40be-861f-c27cee2c6097 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:11.150434 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 574/1097] 10.222.0.18 () {56 vars in 1133 bytes} [Mon Aug 29 20:21:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:21:11.153464 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dbd91e-e0d2-4a91-8d28-48fedd02df42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:11.154907 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dbd91e-e0d2-4a91-8d28-48fedd02df42 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:11.156686 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51016e71-78be-40be-861f-c27cee2c6097 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:11.160260 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:11.229768 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:11.255728 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51016e71-78be-40be-861f-c27cee2c6097 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:11.261613 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 524/1098] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:21:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 20:21:11.262637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dbd91e-e0d2-4a91-8d28-48fedd02df42 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:11.294878 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cab4b-3f74-4d83-8053-08a8e52902b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:11.295492 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cab4b-3f74-4d83-8053-08a8e52902b9 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:11.295776 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cab4b-3f74-4d83-8053-08a8e52902b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:11.296086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cab4b-3f74-4d83-8053-08a8e52902b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:11.367779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cab4b-3f74-4d83-8053-08a8e52902b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:11.370070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cab4b-3f74-4d83-8053-08a8e52902b9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:11.372831 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:11.377182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dbd91e-e0d2-4a91-8d28-48fedd02df42 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:11.411537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a560d1-1055-4327-a753-4a6424c413e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:11.412193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a560d1-1055-4327-a753-4a6424c413e3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:11.412472 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a560d1-1055-4327-a753-4a6424c413e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:11.412742 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a560d1-1055-4327-a753-4a6424c413e3 None admin] PATH_INFO: `/v3/projects/0c4a3bb2e1884623b3a8a3451b2c4c83` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:11.414126 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a560d1-1055-4327-a753-4a6424c413e3 None admin] RBAC: Authorizing `identity:get_project(project_id=0c4a3bb2e1884623b3a8a3451b2c4c83)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:11.416097 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a560d1-1055-4327-a753-4a6424c413e3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:11.417892 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 525/1099] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:21:11 2022] GET /identity/v3/projects/0c4a3bb2e1884623b3a8a3451b2c4c83 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:11.424391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a560d1-1055-4327-a753-4a6424c413e3 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:11.469135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:11.477289 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 575/1100] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:21:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 20:21:11.483673 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cab4b-3f74-4d83-8053-08a8e52902b9 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:11.530843 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a560d1-1055-4327-a753-4a6424c413e3 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:11.569147 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f9108-b4f6-4e62-9cd7-f4df3e52650f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:11.569902 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f9108-b4f6-4e62-9cd7-f4df3e52650f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:11.570292 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f9108-b4f6-4e62-9cd7-f4df3e52650f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:11.570718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f9108-b4f6-4e62-9cd7-f4df3e52650f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:11.571564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f9108-b4f6-4e62-9cd7-f4df3e52650f None admin] RBAC: Adding query filter params (role.id=322da7bd64024770a64bdb73e5c54719, scope.project.id=0c4a3bb2e1884623b3a8a3451b2c4c83, user.id=a78ec1656f174f37936c781e0425485e) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:11.572068 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f9108-b4f6-4e62-9cd7-f4df3e52650f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:11.573629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f9108-b4f6-4e62-9cd7-f4df3e52650f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:11.579795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 526/1101] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:21:11 2022] GET /identity/v3/role_assignments?user.id=a78ec1656f174f37936c781e0425485e&scope.project.id=0c4a3bb2e1884623b3a8a3451b2c4c83&role.id=322da7bd64024770a64bdb73e5c54719 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:11.593886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cab4b-3f74-4d83-8053-08a8e52902b9 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:11.597037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:11.637472 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a9487-6006-40c9-a49f-86a6b9258b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:11.638147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a9487-6006-40c9-a49f-86a6b9258b09 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:11.638430 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a9487-6006-40c9-a49f-86a6b9258b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:11.638707 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a9487-6006-40c9-a49f-86a6b9258b09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:11.651721 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a9487-6006-40c9-a49f-86a6b9258b09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:11.653281 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a9487-6006-40c9-a49f-86a6b9258b09 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:11.670781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-f47a9487-6006-40c9-a49f-86a6b9258b09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:21:11.670781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:11.670781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:11.670781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:11.670781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:11.670781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:11.670781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:11.670781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:11.670781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:11.670781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:11.670781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:11.670781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:11.670781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:21:11.670781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:21:11.670781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:21:11.670781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:21:11.671968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:11.671968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:11.671968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:11.671968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:11.671968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:11.671968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:11.671968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:11.671968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:11.671968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:11.671968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:11.671968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:11.671968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:11.671968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:11.671968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:11.671968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:11.671968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:11.671968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:21:11.672845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:21:11.672845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:21:11.672845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:21:11.672845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 576/1102] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:21:11.677954 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f9108-b4f6-4e62-9cd7-f4df3e52650f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:11.790869 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f9108-b4f6-4e62-9cd7-f4df3e52650f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:11.798397 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:11.830279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee03a08-9547-4644-9846-8c18f274e6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:11.830890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee03a08-9547-4644-9846-8c18f274e6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:11.831167 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee03a08-9547-4644-9846-8c18f274e6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:11.831440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee03a08-9547-4644-9846-8c18f274e6c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:11.832126 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee03a08-9547-4644-9846-8c18f274e6c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:11.832435 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee03a08-9547-4644-9846-8c18f274e6c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:11.833806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee03a08-9547-4644-9846-8c18f274e6c2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:11.853493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 527/1103] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:11.859530 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a9487-6006-40c9-a49f-86a6b9258b09 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:11.962851 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a9487-6006-40c9-a49f-86a6b9258b09 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:11.998146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e44a4-4a92-4034-ac99-df7b1bef41e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:11.998779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e44a4-4a92-4034-ac99-df7b1bef41e7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:11.999058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e44a4-4a92-4034-ac99-df7b1bef41e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:11.999326 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e44a4-4a92-4034-ac99-df7b1bef41e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:12.005984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e44a4-4a92-4034-ac99-df7b1bef41e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:12.007425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e44a4-4a92-4034-ac99-df7b1bef41e7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:12.019258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-6f9e44a4-4a92-4034-ac99-df7b1bef41e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:21:12.019258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:12.019258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:12.019258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:12.019258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:12.019258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:12.019258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:12.019258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:12.019258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:12.019258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:12.019258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:12.019258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:12.019258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:21:12.019258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:21:12.019258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:21:12.020239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:21:12.020239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:12.020239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:12.020239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:12.020239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:12.020239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:12.020239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:12.020239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:12.020239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:12.020239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:12.020239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:12.020239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:12.020239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:12.020239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:12.020239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:12.020239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:12.020239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:12.021459 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:21:12.021459 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:21:12.021459 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:21:12.021459 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:21:12.021459 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:21:12.021459 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:21:12.021459 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:21:12.021459 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:21:12.021459 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 577/1104] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:21:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:21:12.026343 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee03a08-9547-4644-9846-8c18f274e6c2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:12.130161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee03a08-9547-4644-9846-8c18f274e6c2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:12.168576 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443a15f-86fe-4738-a43a-ed09a51cd423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:12.169217 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443a15f-86fe-4738-a43a-ed09a51cd423 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:12.169509 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443a15f-86fe-4738-a43a-ed09a51cd423 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:12.169763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443a15f-86fe-4738-a43a-ed09a51cd423 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:12.170463 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443a15f-86fe-4738-a43a-ed09a51cd423 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:12.170764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443a15f-86fe-4738-a43a-ed09a51cd423 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:12.172226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443a15f-86fe-4738-a43a-ed09a51cd423 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:12.192504 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 528/1105] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:21:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:12.193747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e44a4-4a92-4034-ac99-df7b1bef41e7 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:12.213391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:12.314719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e44a4-4a92-4034-ac99-df7b1bef41e7 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:12.350177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4ca16-7b73-4f0f-a9c1-4d7e2ef5d322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:12.350961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4ca16-7b73-4f0f-a9c1-4d7e2ef5d322 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:12.351334 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4ca16-7b73-4f0f-a9c1-4d7e2ef5d322 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:12.351707 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4ca16-7b73-4f0f-a9c1-4d7e2ef5d322 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:12.352736 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4ca16-7b73-4f0f-a9c1-4d7e2ef5d322 None admin] RBAC: Adding query filter params (role.id=8ece00b091b347179365b72919fdd7dc, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=c0c44754a1c04a2b87081aba6cb509d3) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:12.353751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4ca16-7b73-4f0f-a9c1-4d7e2ef5d322 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:12.355606 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b4ca16-7b73-4f0f-a9c1-4d7e2ef5d322 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:12.363503 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 578/1106] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:21:12 2022] GET /identity/v3/role_assignments?user.id=c0c44754a1c04a2b87081aba6cb509d3&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=8ece00b091b347179365b72919fdd7dc => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:13.584714 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c3f95-b87f-4d9e-89eb-00d1d557d1b2 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:13.585241 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c3f95-b87f-4d9e-89eb-00d1d557d1b2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:13.585241 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c3f95-b87f-4d9e-89eb-00d1d557d1b2 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:13.586532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 529/1107] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:13.592055 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97c604-5962-435d-a724-9d1746ff60b6 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:13.592525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97c604-5962-435d-a724-9d1746ff60b6 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:13.592631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97c604-5962-435d-a724-9d1746ff60b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:13.611087 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-fd97c604-5962-435d-a724-9d1746ff60b6 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:13.612373 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-fd97c604-5962-435d-a724-9d1746ff60b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:13.678982 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 579/1108] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:13.949036 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c3f95-b87f-4d9e-89eb-00d1d557d1b2 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:13.976633 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:14.050290 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c3f95-b87f-4d9e-89eb-00d1d557d1b2 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:14.084067 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396cf10-d299-4886-a825-b8ac9aec7a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:14.084713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396cf10-d299-4886-a825-b8ac9aec7a96 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:14.084988 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396cf10-d299-4886-a825-b8ac9aec7a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:14.085255 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396cf10-d299-4886-a825-b8ac9aec7a96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:14.086363 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396cf10-d299-4886-a825-b8ac9aec7a96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:14.087814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396cf10-d299-4886-a825-b8ac9aec7a96 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:14.089407 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 530/1109] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:21:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:14.095189 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97c604-5962-435d-a724-9d1746ff60b6 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:14.106767 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:14.192546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97c604-5962-435d-a724-9d1746ff60b6 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:14.227552 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0f401-aa2e-4be3-b4e5-2be28b5ba250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:14.228166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0f401-aa2e-4be3-b4e5-2be28b5ba250 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:14.228440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0f401-aa2e-4be3-b4e5-2be28b5ba250 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:14.228708 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0f401-aa2e-4be3-b4e5-2be28b5ba250 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:14.229375 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0f401-aa2e-4be3-b4e5-2be28b5ba250 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:14.230796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc0f401-aa2e-4be3-b4e5-2be28b5ba250 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:14.253846 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 580/1110] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 29 20:21:14 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:16.265388 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a19122-6ada-46d7-b486-1504e63f344a None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:16.265951 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a19122-6ada-46d7-b486-1504e63f344a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:16.265951 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a19122-6ada-46d7-b486-1504e63f344a None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:16.267266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 531/1111] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:16.274017 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125cf75-9a15-42f0-a1c6-b2993d0fe2aa None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:16.274427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125cf75-9a15-42f0-a1c6-b2993d0fe2aa None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:16.274558 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a125cf75-9a15-42f0-a1c6-b2993d0fe2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:16.295393 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a125cf75-9a15-42f0-a1c6-b2993d0fe2aa None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:16.296601 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a125cf75-9a15-42f0-a1c6-b2993d0fe2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:16.357446 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:16.365802 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 581/1112] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:16.639043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a19122-6ada-46d7-b486-1504e63f344a None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:16.736152 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:16.744207 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a19122-6ada-46d7-b486-1504e63f344a None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:16.777440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2d821-b380-43af-af93-88f5c4e5b2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:16.778055 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2d821-b380-43af-af93-88f5c4e5b2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:16.778337 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2d821-b380-43af-af93-88f5c4e5b2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:16.778607 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2d821-b380-43af-af93-88f5c4e5b2a2 None admin] PATH_INFO: `/v3/roles/578c9a17c0fa434f9602f5d2b3648785` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:16.780017 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c2d821-b380-43af-af93-88f5c4e5b2a2 None admin] RBAC: Authorizing `identity:get_role(role_id=578c9a17c0fa434f9602f5d2b3648785)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:16.781384 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c2d821-b380-43af-af93-88f5c4e5b2a2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:16.782704 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 532/1113] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:21:16 2022] GET /identity/v3/roles/578c9a17c0fa434f9602f5d2b3648785 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:16.788763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125cf75-9a15-42f0-a1c6-b2993d0fe2aa None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:16.888562 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a125cf75-9a15-42f0-a1c6-b2993d0fe2aa None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:16.922278 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0396911-b4c9-4159-bfdc-d6ba809bd2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:16.922845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0396911-b4c9-4159-bfdc-d6ba809bd2ab None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:16.923127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0396911-b4c9-4159-bfdc-d6ba809bd2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:16.923403 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0396911-b4c9-4159-bfdc-d6ba809bd2ab None admin] PATH_INFO: `/v3/users/cbca05a1f20d4c84939b3f01c67276ac` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:16.936301 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0396911-b4c9-4159-bfdc-d6ba809bd2ab None admin] RBAC: Authorizing `identity:get_user(user_id=cbca05a1f20d4c84939b3f01c67276ac)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:16.937778 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0396911-b4c9-4159-bfdc-d6ba809bd2ab None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:16.939376 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 582/1114] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:21:16 2022] GET /identity/v3/users/cbca05a1f20d4c84939b3f01c67276ac => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:16.945443 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2d821-b380-43af-af93-88f5c4e5b2a2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:17.045926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2d821-b380-43af-af93-88f5c4e5b2a2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:17.079674 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1549057-660c-48e8-804b-53e2513678eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:17.080296 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1549057-660c-48e8-804b-53e2513678eb None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:17.080572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1549057-660c-48e8-804b-53e2513678eb None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:17.080840 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1549057-660c-48e8-804b-53e2513678eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:17.110263 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:17.146717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1549057-660c-48e8-804b-53e2513678eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:17.148200 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1549057-660c-48e8-804b-53e2513678eb None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:17.244651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 533/1115] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:21:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 20:21:17.250543 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0396911-b4c9-4159-bfdc-d6ba809bd2ab None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:17.306876 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:17.350408 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0396911-b4c9-4159-bfdc-d6ba809bd2ab None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:17.383235 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db2a20-a64d-4d8a-a963-0d1905f4ec16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:17.384429 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db2a20-a64d-4d8a-a963-0d1905f4ec16 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:17.384711 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db2a20-a64d-4d8a-a963-0d1905f4ec16 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:17.384980 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db2a20-a64d-4d8a-a963-0d1905f4ec16 None admin] PATH_INFO: `/v3/projects/0c4a3bb2e1884623b3a8a3451b2c4c83` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:17.386102 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db2a20-a64d-4d8a-a963-0d1905f4ec16 None admin] RBAC: Authorizing `identity:get_project(project_id=0c4a3bb2e1884623b3a8a3451b2c4c83)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:17.387526 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db2a20-a64d-4d8a-a963-0d1905f4ec16 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:17.389193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 583/1116] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:21:17 2022] GET /identity/v3/projects/0c4a3bb2e1884623b3a8a3451b2c4c83 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:17.395423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1549057-660c-48e8-804b-53e2513678eb None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:17.457110 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:17.497373 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1549057-660c-48e8-804b-53e2513678eb None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:17.531422 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d6558-57b0-497d-9c18-78806fd5202d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:17.532048 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d6558-57b0-497d-9c18-78806fd5202d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:17.532331 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d6558-57b0-497d-9c18-78806fd5202d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:17.532604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d6558-57b0-497d-9c18-78806fd5202d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:17.533326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d6558-57b0-497d-9c18-78806fd5202d None admin] RBAC: Adding query filter params (role.id=578c9a17c0fa434f9602f5d2b3648785, scope.project.id=0c4a3bb2e1884623b3a8a3451b2c4c83, user.id=cbca05a1f20d4c84939b3f01c67276ac) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:17.533632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d6558-57b0-497d-9c18-78806fd5202d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:17.535060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d6558-57b0-497d-9c18-78806fd5202d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:17.540767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 534/1117] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:21:17 2022] GET /identity/v3/role_assignments?user.id=cbca05a1f20d4c84939b3f01c67276ac&scope.project.id=0c4a3bb2e1884623b3a8a3451b2c4c83&role.id=578c9a17c0fa434f9602f5d2b3648785 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:19.445413 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8490367-d950-4b3a-8770-95cccb024892 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:19.445997 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8490367-d950-4b3a-8770-95cccb024892 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:19.445997 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8490367-d950-4b3a-8770-95cccb024892 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:19.447884 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 584/1118] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:19.453124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c98f3-ca0e-4c13-91f6-657ca9b25a46 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:19.453554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c98f3-ca0e-4c13-91f6-657ca9b25a46 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:19.453651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c98f3-ca0e-4c13-91f6-657ca9b25a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:19.471895 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-7b5c98f3-ca0e-4c13-91f6-657ca9b25a46 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:19.473118 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7b5c98f3-ca0e-4c13-91f6-657ca9b25a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:19.549878 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 535/1119] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:19.812869 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8490367-d950-4b3a-8770-95cccb024892 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:19.925226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8490367-d950-4b3a-8770-95cccb024892 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:19.948121 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:19.959712 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929c20c-20af-4020-a19c-3e1b337779c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:19.960382 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929c20c-20af-4020-a19c-3e1b337779c3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:19.960663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929c20c-20af-4020-a19c-3e1b337779c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:19.960934 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929c20c-20af-4020-a19c-3e1b337779c3 None admin] PATH_INFO: `/v3/roles/578c9a17c0fa434f9602f5d2b3648785` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:19.962346 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1929c20c-20af-4020-a19c-3e1b337779c3 None admin] RBAC: Authorizing `identity:get_role(role_id=578c9a17c0fa434f9602f5d2b3648785)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:19.963705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1929c20c-20af-4020-a19c-3e1b337779c3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:19.965070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 585/1120] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:21:19 2022] GET /identity/v3/roles/578c9a17c0fa434f9602f5d2b3648785 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:19.970977 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c98f3-ca0e-4c13-91f6-657ca9b25a46 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:20.025923 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:20.070691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c98f3-ca0e-4c13-91f6-657ca9b25a46 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:20.107344 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c40a4d-ff7d-4194-b358-91b9275518fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:20.108006 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c40a4d-ff7d-4194-b358-91b9275518fd None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:20.108287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c40a4d-ff7d-4194-b358-91b9275518fd None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:20.108556 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c40a4d-ff7d-4194-b358-91b9275518fd None admin] PATH_INFO: `/v3/users/cbca05a1f20d4c84939b3f01c67276ac` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:20.109675 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c40a4d-ff7d-4194-b358-91b9275518fd None admin] RBAC: Authorizing `identity:get_user(user_id=cbca05a1f20d4c84939b3f01c67276ac)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:20.111104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c40a4d-ff7d-4194-b358-91b9275518fd None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:20.112728 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 536/1121] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:21:19 2022] GET /identity/v3/users/cbca05a1f20d4c84939b3f01c67276ac => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:20.119127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929c20c-20af-4020-a19c-3e1b337779c3 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:20.216356 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929c20c-20af-4020-a19c-3e1b337779c3 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:20.249040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573ff7d-6586-4697-9d44-8cceff4901e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:20.249666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573ff7d-6586-4697-9d44-8cceff4901e1 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:20.249945 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573ff7d-6586-4697-9d44-8cceff4901e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:20.250217 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573ff7d-6586-4697-9d44-8cceff4901e1 None admin] PATH_INFO: `/v3/projects/0c4a3bb2e1884623b3a8a3451b2c4c83` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:20.251308 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f573ff7d-6586-4697-9d44-8cceff4901e1 None admin] RBAC: Authorizing `identity:get_project(project_id=0c4a3bb2e1884623b3a8a3451b2c4c83)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:20.252750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f573ff7d-6586-4697-9d44-8cceff4901e1 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:20.254343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 586/1122] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:21:20 2022] GET /identity/v3/projects/0c4a3bb2e1884623b3a8a3451b2c4c83 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:20.260943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c40a4d-ff7d-4194-b358-91b9275518fd None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:20.358475 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c40a4d-ff7d-4194-b358-91b9275518fd None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:20.381169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:20.394855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0253c6-c198-42c3-9744-89dec50ba74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:20.395501 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0253c6-c198-42c3-9744-89dec50ba74b None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:20.395785 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0253c6-c198-42c3-9744-89dec50ba74b None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:20.396095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0253c6-c198-42c3-9744-89dec50ba74b None admin] PATH_INFO: `/v3/projects/0c4a3bb2e1884623b3a8a3451b2c4c83/users/cbca05a1f20d4c84939b3f01c67276ac/roles/578c9a17c0fa434f9602f5d2b3648785` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:20.398217 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0253c6-c198-42c3-9744-89dec50ba74b None admin] RBAC: Authorizing `identity:create_grant(project_id=0c4a3bb2e1884623b3a8a3451b2c4c83, user_id=cbca05a1f20d4c84939b3f01c67276ac, role_id=578c9a17c0fa434f9602f5d2b3648785)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:20.399648 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0253c6-c198-42c3-9744-89dec50ba74b None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:20.420174 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 537/1123] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:21:20 2022] PUT /identity/v3/projects/0c4a3bb2e1884623b3a8a3451b2c4c83/users/cbca05a1f20d4c84939b3f01c67276ac/roles/578c9a17c0fa434f9602f5d2b3648785 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:22.368712 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417571e-1b43-4715-b9b5-61bb42b452be None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:22.369269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417571e-1b43-4715-b9b5-61bb42b452be None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:22.369269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417571e-1b43-4715-b9b5-61bb42b452be None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:22.370603 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 587/1124] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:22.376197 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91072f9-843e-4d88-ac04-763bf4780dfd None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:22.376624 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91072f9-843e-4d88-ac04-763bf4780dfd None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:22.376751 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91072f9-843e-4d88-ac04-763bf4780dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:22.397226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c91072f9-843e-4d88-ac04-763bf4780dfd None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:22.398427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c91072f9-843e-4d88-ac04-763bf4780dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:22.472880 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 538/1125] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:22.746101 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417571e-1b43-4715-b9b5-61bb42b452be None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:22.824742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:22.880248 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417571e-1b43-4715-b9b5-61bb42b452be None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:22.919324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363112ca-feea-4b4c-b3f1-6d374867075d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:22.919988 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363112ca-feea-4b4c-b3f1-6d374867075d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:22.920300 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363112ca-feea-4b4c-b3f1-6d374867075d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:22.920581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363112ca-feea-4b4c-b3f1-6d374867075d None admin] PATH_INFO: `/v3/roles/578c9a17c0fa434f9602f5d2b3648785` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:22.921998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363112ca-feea-4b4c-b3f1-6d374867075d None admin] RBAC: Authorizing `identity:get_role(role_id=578c9a17c0fa434f9602f5d2b3648785)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:22.923382 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363112ca-feea-4b4c-b3f1-6d374867075d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:22.924794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 588/1126] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:21:22 2022] GET /identity/v3/roles/578c9a17c0fa434f9602f5d2b3648785 => generated 223 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:22.930818 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91072f9-843e-4d88-ac04-763bf4780dfd None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:23.033901 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91072f9-843e-4d88-ac04-763bf4780dfd None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:23.048773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:23.070480 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36c42f-8d0e-49bb-9e53-99326dbe271f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:23.071160 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36c42f-8d0e-49bb-9e53-99326dbe271f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:23.071958 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36c42f-8d0e-49bb-9e53-99326dbe271f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:23.072243 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36c42f-8d0e-49bb-9e53-99326dbe271f None admin] PATH_INFO: `/v3/users/cbca05a1f20d4c84939b3f01c67276ac` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:23.073407 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36c42f-8d0e-49bb-9e53-99326dbe271f None admin] RBAC: Authorizing `identity:get_user(user_id=cbca05a1f20d4c84939b3f01c67276ac)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:23.074856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36c42f-8d0e-49bb-9e53-99326dbe271f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:23.076554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 539/1127] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:21:22 2022] GET /identity/v3/users/cbca05a1f20d4c84939b3f01c67276ac => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:23.082504 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363112ca-feea-4b4c-b3f1-6d374867075d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:23.190851 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363112ca-feea-4b4c-b3f1-6d374867075d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:23.217114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:23.226373 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ae882-6929-4466-8715-3f59648059b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:23.227007 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ae882-6929-4466-8715-3f59648059b2 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:23.227285 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ae882-6929-4466-8715-3f59648059b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:23.227556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ae882-6929-4466-8715-3f59648059b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:23.296390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ae882-6929-4466-8715-3f59648059b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:23.297738 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ae882-6929-4466-8715-3f59648059b2 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:23.393726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 589/1128] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:21:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:21:23.399731 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36c42f-8d0e-49bb-9e53-99326dbe271f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:23.500167 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36c42f-8d0e-49bb-9e53-99326dbe271f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:23.533077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfdc8e8-9ba4-4a1e-b18d-9a768dc4acc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:23.533693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdc8e8-9ba4-4a1e-b18d-9a768dc4acc2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:23.533974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdc8e8-9ba4-4a1e-b18d-9a768dc4acc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:23.534249 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdc8e8-9ba4-4a1e-b18d-9a768dc4acc2 None admin] PATH_INFO: `/v3/projects/0c4a3bb2e1884623b3a8a3451b2c4c83` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:23.535351 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfdc8e8-9ba4-4a1e-b18d-9a768dc4acc2 None admin] RBAC: Authorizing `identity:get_project(project_id=0c4a3bb2e1884623b3a8a3451b2c4c83)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:23.536802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfdc8e8-9ba4-4a1e-b18d-9a768dc4acc2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:23.538702 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 540/1129] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:21:23 2022] GET /identity/v3/projects/0c4a3bb2e1884623b3a8a3451b2c4c83 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:23.545405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ae882-6929-4466-8715-3f59648059b2 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:23.572138 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:23.643147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ae882-6929-4466-8715-3f59648059b2 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:23.676917 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbbc08e-3ae2-4f69-8755-ce0ccbc8cd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:23.677520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbc08e-3ae2-4f69-8755-ce0ccbc8cd85 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:23.677808 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbc08e-3ae2-4f69-8755-ce0ccbc8cd85 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:23.678080 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbbc08e-3ae2-4f69-8755-ce0ccbc8cd85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:23.678808 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbbc08e-3ae2-4f69-8755-ce0ccbc8cd85 None admin] RBAC: Adding query filter params (role.id=578c9a17c0fa434f9602f5d2b3648785, scope.project.id=0c4a3bb2e1884623b3a8a3451b2c4c83, user.id=cbca05a1f20d4c84939b3f01c67276ac) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:23.679117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbbc08e-3ae2-4f69-8755-ce0ccbc8cd85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:23.680596 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbbc08e-3ae2-4f69-8755-ce0ccbc8cd85 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:23.686264 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 590/1130] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:21:23 2022] GET /identity/v3/role_assignments?user.id=cbca05a1f20d4c84939b3f01c67276ac&scope.project.id=0c4a3bb2e1884623b3a8a3451b2c4c83&role.id=578c9a17c0fa434f9602f5d2b3648785 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:25.595333 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389042cb-42fc-419c-807a-a254c2108bf0 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:25.595854 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389042cb-42fc-419c-807a-a254c2108bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:25.595854 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389042cb-42fc-419c-807a-a254c2108bf0 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:25.597182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 541/1131] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:25.602789 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b36c49-5455-4516-b8a2-282578107ede None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:25.603205 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b36c49-5455-4516-b8a2-282578107ede None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:25.603365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b36c49-5455-4516-b8a2-282578107ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:25.621064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-59b36c49-5455-4516-b8a2-282578107ede None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:25.622263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-59b36c49-5455-4516-b8a2-282578107ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:25.686379 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 591/1132] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:25.952839 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389042cb-42fc-419c-807a-a254c2108bf0 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:26.002971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:26.052024 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389042cb-42fc-419c-807a-a254c2108bf0 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:26.085175 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46966fe6-a9ba-4c82-b1e4-71abf2438ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:26.085798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46966fe6-a9ba-4c82-b1e4-71abf2438ebe None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:26.086166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46966fe6-a9ba-4c82-b1e4-71abf2438ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:26.086360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46966fe6-a9ba-4c82-b1e4-71abf2438ebe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:26.087469 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46966fe6-a9ba-4c82-b1e4-71abf2438ebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:26.088936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46966fe6-a9ba-4c82-b1e4-71abf2438ebe None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:26.090496 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 542/1133] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:21:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:26.096279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b36c49-5455-4516-b8a2-282578107ede None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:26.116312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:26.191381 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b36c49-5455-4516-b8a2-282578107ede None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:26.223820 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773a3b0-a7da-49ad-8be3-ccf38dc870e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:26.224381 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773a3b0-a7da-49ad-8be3-ccf38dc870e0 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:26.224659 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773a3b0-a7da-49ad-8be3-ccf38dc870e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:26.224929 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773a3b0-a7da-49ad-8be3-ccf38dc870e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:26.225569 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773a3b0-a7da-49ad-8be3-ccf38dc870e0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:26.227028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773a3b0-a7da-49ad-8be3-ccf38dc870e0 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:26.253164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 592/1134] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 29 20:21:26 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:28.174618 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353a650-7226-411a-9d66-c25a12b954f5 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:28.175210 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353a650-7226-411a-9d66-c25a12b954f5 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:28.175210 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353a650-7226-411a-9d66-c25a12b954f5 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:28.176536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 543/1135] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:28.182365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd78fb3-848a-44af-b600-bc394000087a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:28.182809 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd78fb3-848a-44af-b600-bc394000087a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:28.182919 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd78fb3-848a-44af-b600-bc394000087a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:28.200714 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-3cd78fb3-848a-44af-b600-bc394000087a None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:28.201888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3cd78fb3-848a-44af-b600-bc394000087a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:28.266136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 593/1136] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:28.534874 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353a650-7226-411a-9d66-c25a12b954f5 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:28.635917 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353a650-7226-411a-9d66-c25a12b954f5 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:28.654741 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:28.671851 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56aa1f-0995-415d-bb0d-8bec464b7186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:28.672503 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56aa1f-0995-415d-bb0d-8bec464b7186 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:28.672786 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56aa1f-0995-415d-bb0d-8bec464b7186 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:28.673060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56aa1f-0995-415d-bb0d-8bec464b7186 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:28.674158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56aa1f-0995-415d-bb0d-8bec464b7186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:28.675586 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56aa1f-0995-415d-bb0d-8bec464b7186 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:28.677196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 544/1137] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:21:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:28.682958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd78fb3-848a-44af-b600-bc394000087a None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:28.685156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:28.781602 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd78fb3-848a-44af-b600-bc394000087a None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:28.813913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853a7ec-4b9f-45c8-ae03-801b50be6180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:28.814458 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853a7ec-4b9f-45c8-ae03-801b50be6180 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:28.814736 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853a7ec-4b9f-45c8-ae03-801b50be6180 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:28.815005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853a7ec-4b9f-45c8-ae03-801b50be6180 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:28.815646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853a7ec-4b9f-45c8-ae03-801b50be6180 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:28.817096 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853a7ec-4b9f-45c8-ae03-801b50be6180 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:28.838918 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 594/1138] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 29 20:21:28 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:30.786076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac5dfd-ad08-423a-b479-d5f07e64bef4 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:30.786618 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac5dfd-ad08-423a-b479-d5f07e64bef4 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:30.786718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac5dfd-ad08-423a-b479-d5f07e64bef4 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:30.788305 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 545/1139] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:30.793373 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c31036-49eb-497b-88d0-568e2d3e93dd None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:30.793811 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c31036-49eb-497b-88d0-568e2d3e93dd None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:30.793916 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c31036-49eb-497b-88d0-568e2d3e93dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:30.812068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-99c31036-49eb-497b-88d0-568e2d3e93dd None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:30.813233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-99c31036-49eb-497b-88d0-568e2d3e93dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:30.863454 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:30.878323 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 595/1140] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:31.160338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac5dfd-ad08-423a-b479-d5f07e64bef4 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:31.259249 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac5dfd-ad08-423a-b479-d5f07e64bef4 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:31.293469 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d3782-6e2a-4f69-8de9-8f2b876a1608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:31.294102 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d3782-6e2a-4f69-8de9-8f2b876a1608 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:31.294383 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d3782-6e2a-4f69-8de9-8f2b876a1608 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:31.294659 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d3782-6e2a-4f69-8de9-8f2b876a1608 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:31.305134 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d3782-6e2a-4f69-8de9-8f2b876a1608 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:21:31.309599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-c66d3782-6e2a-4f69-8de9-8f2b876a1608 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:21:31.309599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:31.309599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:31.309599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:31.309599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:31.309599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:31.309599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:31.309599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:31.309599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:31.309599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:31.309599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:31.309599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:31.309599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:21:31.309599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:21:31.309599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:21:31.309599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:21:31.310767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:21:31.310767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:21:31.310767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:21:31.310767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:21:31.310767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:31.310767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:31.310767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:31.310767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:31.310767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:31.310767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:31.310767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:31.310767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:31.310767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:31.310767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:31.310767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:31.310767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:31.311838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:31.311838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:31.311838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:31.311838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:31.311838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:21:31.311838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:21:31.311838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:21:31.311838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:21:31.311838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:21:31.311838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:21:31.311838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:21:31.311838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:21:31.311838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 546/1141] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:21:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:21:31.316800 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c31036-49eb-497b-88d0-568e2d3e93dd None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:31.411346 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c31036-49eb-497b-88d0-568e2d3e93dd None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:31.443781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716900d-31b0-4db9-9bc4-268f2fa12150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:31.444402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716900d-31b0-4db9-9bc4-268f2fa12150 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:31.444681 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716900d-31b0-4db9-9bc4-268f2fa12150 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:31.444952 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716900d-31b0-4db9-9bc4-268f2fa12150 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:31.445612 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5716900d-31b0-4db9-9bc4-268f2fa12150 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:31.445910 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5716900d-31b0-4db9-9bc4-268f2fa12150 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:31.447252 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5716900d-31b0-4db9-9bc4-268f2fa12150 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:31.454996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 596/1142] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:21:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:21:31.461279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d3782-6e2a-4f69-8de9-8f2b876a1608 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:31.509457 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:31.587677 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d3782-6e2a-4f69-8de9-8f2b876a1608 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:31.620373 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df2b3e-cc95-4d57-a905-91393b4bc720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:31.620995 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df2b3e-cc95-4d57-a905-91393b4bc720 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:31.621274 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df2b3e-cc95-4d57-a905-91393b4bc720 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:31.621544 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df2b3e-cc95-4d57-a905-91393b4bc720 None admin] PATH_INFO: `/v3/users/da24716eade94e0f928d4b744d4640c1` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:31.634754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df2b3e-cc95-4d57-a905-91393b4bc720 None admin] RBAC: Authorizing `identity:get_user(user_id=da24716eade94e0f928d4b744d4640c1)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:31.636215 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df2b3e-cc95-4d57-a905-91393b4bc720 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:31.637811 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 547/1143] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:21:31 2022] GET /identity/v3/users/da24716eade94e0f928d4b744d4640c1 => generated 286 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:31.644068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716900d-31b0-4db9-9bc4-268f2fa12150 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:31.693328 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:31.742678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716900d-31b0-4db9-9bc4-268f2fa12150 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:31.775883 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c210d-61b8-430e-99f6-f0de914888ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:31.776510 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c210d-61b8-430e-99f6-f0de914888ee None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:31.776790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c210d-61b8-430e-99f6-f0de914888ee None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:31.777065 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c210d-61b8-430e-99f6-f0de914888ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:31.841573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c210d-61b8-430e-99f6-f0de914888ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:31.842998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c210d-61b8-430e-99f6-f0de914888ee None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:31.889731 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:31.947569 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 597/1144] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:21:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:21:31.953578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df2b3e-cc95-4d57-a905-91393b4bc720 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:32.049691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df2b3e-cc95-4d57-a905-91393b4bc720 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:32.051999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:32.084441 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65117a6e-f4d2-4c3a-94be-3c23b4ffd952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:32.085067 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65117a6e-f4d2-4c3a-94be-3c23b4ffd952 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:32.085346 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65117a6e-f4d2-4c3a-94be-3c23b4ffd952 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:32.085647 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65117a6e-f4d2-4c3a-94be-3c23b4ffd952 None admin] PATH_INFO: `/v3/projects/77ba770afb84424381247cd10072261c` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:32.086754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65117a6e-f4d2-4c3a-94be-3c23b4ffd952 None admin] RBAC: Authorizing `identity:get_project(project_id=77ba770afb84424381247cd10072261c)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:32.088203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65117a6e-f4d2-4c3a-94be-3c23b4ffd952 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:32.089833 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 548/1145] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:21:31 2022] GET /identity/v3/projects/77ba770afb84424381247cd10072261c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:32.095978 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c210d-61b8-430e-99f6-f0de914888ee None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:32.189895 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c210d-61b8-430e-99f6-f0de914888ee None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:32.222920 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6464bd-85c8-47a1-aaa4-6e676c68619d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:32.223451 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6464bd-85c8-47a1-aaa4-6e676c68619d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:32.223726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6464bd-85c8-47a1-aaa4-6e676c68619d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:32.224018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6464bd-85c8-47a1-aaa4-6e676c68619d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:32.224736 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6464bd-85c8-47a1-aaa4-6e676c68619d None admin] RBAC: Adding query filter params (role.id=322da7bd64024770a64bdb73e5c54719, scope.project.id=77ba770afb84424381247cd10072261c, user.id=da24716eade94e0f928d4b744d4640c1) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:32.225040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6464bd-85c8-47a1-aaa4-6e676c68619d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:32.226584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6464bd-85c8-47a1-aaa4-6e676c68619d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:32.232084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 598/1146] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:21:32 2022] GET /identity/v3/role_assignments?user.id=da24716eade94e0f928d4b744d4640c1&scope.project.id=77ba770afb84424381247cd10072261c&role.id=322da7bd64024770a64bdb73e5c54719 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:34.079526 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f0ce8-2951-4421-99f5-274eadbcc446 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:34.080086 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f0ce8-2951-4421-99f5-274eadbcc446 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:34.080086 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f0ce8-2951-4421-99f5-274eadbcc446 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:34.081452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 549/1147] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:34.086842 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28516860-602b-4e34-86fe-1857d0a44f1b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:34.087250 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28516860-602b-4e34-86fe-1857d0a44f1b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:34.087393 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28516860-602b-4e34-86fe-1857d0a44f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:34.106107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-28516860-602b-4e34-86fe-1857d0a44f1b None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:34.107392 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-28516860-602b-4e34-86fe-1857d0a44f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:34.117448 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:34.173586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 599/1148] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:34.442224 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67f0ce8-2951-4421-99f5-274eadbcc446 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:34.542673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67f0ce8-2951-4421-99f5-274eadbcc446 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:34.575065 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a5cf6-42bf-42ef-89e2-22025fdd429a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:34.575686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a5cf6-42bf-42ef-89e2-22025fdd429a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:34.575991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a5cf6-42bf-42ef-89e2-22025fdd429a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:34.576270 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a5cf6-42bf-42ef-89e2-22025fdd429a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:34.586710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a5cf6-42bf-42ef-89e2-22025fdd429a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:21:34.591495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-2e4a5cf6-42bf-42ef-89e2-22025fdd429a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:21:34.591495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:34.591495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:34.591495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:34.591495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:34.591495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:34.591495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:34.591495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:34.591495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:34.591495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:34.591495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:34.591495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:34.591495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:21:34.591495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:21:34.591495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:21:34.591495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:21:34.592606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:21:34.592606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:21:34.592606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:21:34.592606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:21:34.592606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:34.592606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:34.592606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:34.592606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:34.592606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:34.592606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:34.592606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:34.592606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:34.592606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:34.592606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:34.592606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:34.592606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:34.593637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:34.593637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:34.593637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:34.593637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:34.593637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:21:34.593637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:21:34.593637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:21:34.593637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:21:34.593637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:21:34.593637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:21:34.593637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:21:34.593637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:21:34.593637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 550/1149] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:21:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:21:34.598576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28516860-602b-4e34-86fe-1857d0a44f1b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:34.697195 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28516860-602b-4e34-86fe-1857d0a44f1b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:34.735666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81ffd2-099b-47a7-9aee-6bea23865121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:34.736292 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81ffd2-099b-47a7-9aee-6bea23865121 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:34.736571 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81ffd2-099b-47a7-9aee-6bea23865121 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:34.736842 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81ffd2-099b-47a7-9aee-6bea23865121 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:34.737498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81ffd2-099b-47a7-9aee-6bea23865121 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:34.737798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81ffd2-099b-47a7-9aee-6bea23865121 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:34.739162 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81ffd2-099b-47a7-9aee-6bea23865121 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:34.746760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 600/1150] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:21:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:34.752759 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a5cf6-42bf-42ef-89e2-22025fdd429a None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:34.784676 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:34.852966 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a5cf6-42bf-42ef-89e2-22025fdd429a None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:34.885538 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21bec1-36f5-4014-a7bf-83a80b208cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:34.886160 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21bec1-36f5-4014-a7bf-83a80b208cfb None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:34.886441 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21bec1-36f5-4014-a7bf-83a80b208cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:34.886715 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21bec1-36f5-4014-a7bf-83a80b208cfb None admin] PATH_INFO: `/v3/users/da24716eade94e0f928d4b744d4640c1` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:34.887825 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21bec1-36f5-4014-a7bf-83a80b208cfb None admin] RBAC: Authorizing `identity:get_user(user_id=da24716eade94e0f928d4b744d4640c1)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:34.889286 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21bec1-36f5-4014-a7bf-83a80b208cfb None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:34.890851 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 551/1151] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:21:34 2022] GET /identity/v3/users/da24716eade94e0f928d4b744d4640c1 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:34.896598 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81ffd2-099b-47a7-9aee-6bea23865121 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:34.906374 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:35.007056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81ffd2-099b-47a7-9aee-6bea23865121 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:35.040258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbd6e5-5345-423b-a845-844c6d105385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:35.040880 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbd6e5-5345-423b-a845-844c6d105385 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:35.041161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbd6e5-5345-423b-a845-844c6d105385 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:35.041437 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbd6e5-5345-423b-a845-844c6d105385 None admin] PATH_INFO: `/v3/projects/77ba770afb84424381247cd10072261c` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:35.042530 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbbd6e5-5345-423b-a845-844c6d105385 None admin] RBAC: Authorizing `identity:get_project(project_id=77ba770afb84424381247cd10072261c)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:35.043973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbbd6e5-5345-423b-a845-844c6d105385 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:35.045591 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 601/1152] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:21:34 2022] GET /identity/v3/projects/77ba770afb84424381247cd10072261c => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:35.051564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21bec1-36f5-4014-a7bf-83a80b208cfb None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:35.145748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:35.149443 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21bec1-36f5-4014-a7bf-83a80b208cfb None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:35.183512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0931b4c-3022-4611-ac68-a6218ef5464d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:35.184192 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0931b4c-3022-4611-ac68-a6218ef5464d None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:35.184475 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0931b4c-3022-4611-ac68-a6218ef5464d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:35.184751 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0931b4c-3022-4611-ac68-a6218ef5464d None admin] PATH_INFO: `/v3/projects/77ba770afb84424381247cd10072261c/users/da24716eade94e0f928d4b744d4640c1/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:35.186644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0931b4c-3022-4611-ac68-a6218ef5464d None admin] RBAC: Authorizing `identity:create_grant(project_id=77ba770afb84424381247cd10072261c, user_id=da24716eade94e0f928d4b744d4640c1, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:35.188234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0931b4c-3022-4611-ac68-a6218ef5464d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:35.201538 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 552/1153] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:21:35 2022] PUT /identity/v3/projects/77ba770afb84424381247cd10072261c/users/da24716eade94e0f928d4b744d4640c1/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:21:37.060275 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f448eae-ad22-4946-a389-84418381f4a5 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:37.060805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f448eae-ad22-4946-a389-84418381f4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:37.060805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f448eae-ad22-4946-a389-84418381f4a5 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:37.062165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 602/1154] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:37.067681 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c752a-6292-4276-ab5c-8878f5c5686e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:37.068123 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c752a-6292-4276-ab5c-8878f5c5686e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:37.068260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c752a-6292-4276-ab5c-8878f5c5686e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:37.087965 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ea2c752a-6292-4276-ab5c-8878f5c5686e None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:37.089162 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ea2c752a-6292-4276-ab5c-8878f5c5686e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:37.154558 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 553/1155] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:37.419824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f448eae-ad22-4946-a389-84418381f4a5 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:37.492951 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:37.518298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f448eae-ad22-4946-a389-84418381f4a5 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:37.549748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee00ba-5576-417c-86db-81bf86c9022a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:37.550368 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee00ba-5576-417c-86db-81bf86c9022a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:37.550644 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee00ba-5576-417c-86db-81bf86c9022a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:37.550912 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee00ba-5576-417c-86db-81bf86c9022a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:37.561392 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee00ba-5576-417c-86db-81bf86c9022a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:21:37.566165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-36ee00ba-5576-417c-86db-81bf86c9022a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:21:37.566165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:37.566165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:37.566165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:37.566165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:37.566165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:37.566165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:37.566165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:37.566165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:37.566165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:37.566165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:37.566165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:37.566165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:21:37.566165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:21:37.566165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:21:37.566165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:21:37.567324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:21:37.567324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:21:37.567324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:21:37.567324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:21:37.567324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:37.567324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:37.567324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:37.567324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:37.567324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:37.567324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:37.567324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:37.567324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:37.567324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:37.567324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:37.567324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:37.567324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:37.568257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:37.568257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:37.568257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:37.568257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:37.568257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:21:37.568257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:21:37.568257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:21:37.568257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:21:37.568257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:21:37.568257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:21:37.568257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:21:37.568257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:21:37.568257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 603/1156] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:21:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:21:37.573921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c752a-6292-4276-ab5c-8878f5c5686e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:37.660652 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:37.670793 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c752a-6292-4276-ab5c-8878f5c5686e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:37.703690 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35501e82-0315-4ad7-bddd-58566a1a13c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:37.704379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35501e82-0315-4ad7-bddd-58566a1a13c0 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:37.704606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35501e82-0315-4ad7-bddd-58566a1a13c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:37.704878 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35501e82-0315-4ad7-bddd-58566a1a13c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:37.705570 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35501e82-0315-4ad7-bddd-58566a1a13c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:37.705839 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35501e82-0315-4ad7-bddd-58566a1a13c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:37.707206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35501e82-0315-4ad7-bddd-58566a1a13c0 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:37.716612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 554/1157] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:21:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:37.722523 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee00ba-5576-417c-86db-81bf86c9022a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:37.821402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee00ba-5576-417c-86db-81bf86c9022a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:37.854170 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d9077-ffd3-43ce-87eb-3c0da7187780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:37.854801 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d9077-ffd3-43ce-87eb-3c0da7187780 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:37.855084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d9077-ffd3-43ce-87eb-3c0da7187780 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:37.855360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d9077-ffd3-43ce-87eb-3c0da7187780 None admin] PATH_INFO: `/v3/users/da24716eade94e0f928d4b744d4640c1` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:37.856541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d9077-ffd3-43ce-87eb-3c0da7187780 None admin] RBAC: Authorizing `identity:get_user(user_id=da24716eade94e0f928d4b744d4640c1)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:37.858003 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d9077-ffd3-43ce-87eb-3c0da7187780 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:37.859550 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 604/1158] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:21:37 2022] GET /identity/v3/users/da24716eade94e0f928d4b744d4640c1 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:37.865566 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35501e82-0315-4ad7-bddd-58566a1a13c0 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:37.963876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35501e82-0315-4ad7-bddd-58566a1a13c0 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:37.997277 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66154258-fc54-48e3-9889-120eaacf0101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:37.997878 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66154258-fc54-48e3-9889-120eaacf0101 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:37.998158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66154258-fc54-48e3-9889-120eaacf0101 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:37.998432 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66154258-fc54-48e3-9889-120eaacf0101 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:38.019441 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:38.064669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66154258-fc54-48e3-9889-120eaacf0101 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:38.066108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66154258-fc54-48e3-9889-120eaacf0101 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:38.160908 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 555/1159] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:21:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:21:38.167707 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d9077-ffd3-43ce-87eb-3c0da7187780 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:38.170655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:38.273697 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d9077-ffd3-43ce-87eb-3c0da7187780 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:38.321019 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07acb19-92ff-4665-9c3c-228e4b4aa4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:38.321665 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07acb19-92ff-4665-9c3c-228e4b4aa4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:38.321944 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07acb19-92ff-4665-9c3c-228e4b4aa4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:38.322217 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07acb19-92ff-4665-9c3c-228e4b4aa4f0 None admin] PATH_INFO: `/v3/projects/77ba770afb84424381247cd10072261c` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:38.323358 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07acb19-92ff-4665-9c3c-228e4b4aa4f0 None admin] RBAC: Authorizing `identity:get_project(project_id=77ba770afb84424381247cd10072261c)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:38.324827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07acb19-92ff-4665-9c3c-228e4b4aa4f0 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:38.326481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 605/1160] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:21:38 2022] GET /identity/v3/projects/77ba770afb84424381247cd10072261c => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:38.333056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66154258-fc54-48e3-9889-120eaacf0101 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:38.390370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:38.438298 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66154258-fc54-48e3-9889-120eaacf0101 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:38.471957 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58904f92-1b09-48a1-9f6d-79e138de47af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:38.472580 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58904f92-1b09-48a1-9f6d-79e138de47af None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:38.472858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58904f92-1b09-48a1-9f6d-79e138de47af None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:38.473130 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58904f92-1b09-48a1-9f6d-79e138de47af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:38.473857 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58904f92-1b09-48a1-9f6d-79e138de47af None admin] RBAC: Adding query filter params (role.id=322da7bd64024770a64bdb73e5c54719, scope.project.id=77ba770afb84424381247cd10072261c, user.id=da24716eade94e0f928d4b744d4640c1) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:38.474166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58904f92-1b09-48a1-9f6d-79e138de47af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:38.476263 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58904f92-1b09-48a1-9f6d-79e138de47af None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:38.482591 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 556/1161] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:21:38 2022] GET /identity/v3/role_assignments?user.id=da24716eade94e0f928d4b744d4640c1&scope.project.id=77ba770afb84424381247cd10072261c&role.id=322da7bd64024770a64bdb73e5c54719 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:40.410783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3abf82-4c95-4341-b9d2-5c28628bdd64 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:40.411547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3abf82-4c95-4341-b9d2-5c28628bdd64 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:40.411665 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3abf82-4c95-4341-b9d2-5c28628bdd64 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:40.412980 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 606/1162] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:40.417975 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54f889-2584-484d-a864-6eb3985ed527 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:40.418382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54f889-2584-484d-a864-6eb3985ed527 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:40.418511 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a54f889-2584-484d-a864-6eb3985ed527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:40.436704 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1a54f889-2584-484d-a864-6eb3985ed527 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:40.437876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1a54f889-2584-484d-a864-6eb3985ed527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:40.501736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 557/1163] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:40.780719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3abf82-4c95-4341-b9d2-5c28628bdd64 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:40.848700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:40.879166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3abf82-4c95-4341-b9d2-5c28628bdd64 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:40.912020 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47814681-71c3-47f1-9b66-464fc08d6938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:40.912614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47814681-71c3-47f1-9b66-464fc08d6938 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:40.912926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47814681-71c3-47f1-9b66-464fc08d6938 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:40.913400 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47814681-71c3-47f1-9b66-464fc08d6938 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:40.978655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47814681-71c3-47f1-9b66-464fc08d6938 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:40.980129 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47814681-71c3-47f1-9b66-464fc08d6938 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:41.050427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:41.087027 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 607/1164] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:21:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:21:41.093174 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a54f889-2584-484d-a864-6eb3985ed527 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:41.140053 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:41.194692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a54f889-2584-484d-a864-6eb3985ed527 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:41.226823 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9343a-1fd0-4e5e-91c5-742602a53be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:41.227435 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9343a-1fd0-4e5e-91c5-742602a53be0 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:41.227714 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9343a-1fd0-4e5e-91c5-742602a53be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:41.228011 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9343a-1fd0-4e5e-91c5-742602a53be0 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:41.234426 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e9343a-1fd0-4e5e-91c5-742602a53be0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:41.235852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e9343a-1fd0-4e5e-91c5-742602a53be0 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:41.247430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-63e9343a-1fd0-4e5e-91c5-742602a53be0 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 20:21:41.247430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:41.247430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 29 20:21:41.247430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 29 20:21:41.247430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:21:41.247430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:21:41.247430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:21:41.247430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:21:41.247430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 29 20:21:41.247430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:21:41.247430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:21:41.247430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:21:41.247430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:41.247430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:41.247430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:41.247430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:41.247430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:41.248711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:41.248711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:41.248711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:41.248711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:41.248711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:41.248711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:41.248711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:21:41.248711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:21:41.248711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:21:41.248711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:21:41.248711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:41.248711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:41.248711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:41.248711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:41.248711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:41.248711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:41.249775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:41.249775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:41.249775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:41.249775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:41.249775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:41.249775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:41.249775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:41.249775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:41.249775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:41.249775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:41.249775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 29 20:21:41.249775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:21:41.249775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 29 20:21:41.249775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:21:41.249775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 558/1165] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Aug 29 20:21:41 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:21:41.254666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47814681-71c3-47f1-9b66-464fc08d6938 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:41.352441 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47814681-71c3-47f1-9b66-464fc08d6938 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:41.385345 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de29f6b-70a5-41b8-bf22-4432d9a7568b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:41.385947 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de29f6b-70a5-41b8-bf22-4432d9a7568b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:41.386231 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de29f6b-70a5-41b8-bf22-4432d9a7568b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:41.386506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de29f6b-70a5-41b8-bf22-4432d9a7568b None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:41.387141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de29f6b-70a5-41b8-bf22-4432d9a7568b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:41.387446 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de29f6b-70a5-41b8-bf22-4432d9a7568b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:41.388823 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de29f6b-70a5-41b8-bf22-4432d9a7568b None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:41.395609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 608/1166] 10.222.0.18 () {56 vars in 1115 bytes} [Mon Aug 29 20:21:41 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:43.283256 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec313e-aa33-44d3-add0-03946e4bb1f8 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:43.283790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec313e-aa33-44d3-add0-03946e4bb1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:43.283790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec313e-aa33-44d3-add0-03946e4bb1f8 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:43.285242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 559/1167] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:43.291336 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb09d9-1db5-46e8-b265-aa2fa5a7d7f5 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:43.291800 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb09d9-1db5-46e8-b265-aa2fa5a7d7f5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:43.291911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb09d9-1db5-46e8-b265-aa2fa5a7d7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:43.310673 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d4fb09d9-1db5-46e8-b265-aa2fa5a7d7f5 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:43.311873 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d4fb09d9-1db5-46e8-b265-aa2fa5a7d7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:43.339039 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:43.378822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 609/1168] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:43.652247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec313e-aa33-44d3-add0-03946e4bb1f8 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:43.754928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec313e-aa33-44d3-add0-03946e4bb1f8 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:43.763257 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:43.791063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db770e0-ed3d-4afa-8bc7-35d3ede520c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:43.791660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db770e0-ed3d-4afa-8bc7-35d3ede520c9 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:43.791968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db770e0-ed3d-4afa-8bc7-35d3ede520c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:43.792235 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db770e0-ed3d-4afa-8bc7-35d3ede520c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:43.792908 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db770e0-ed3d-4afa-8bc7-35d3ede520c9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:43.794275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db770e0-ed3d-4afa-8bc7-35d3ede520c9 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:43.811074 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 560/1169] 10.222.0.18 () {60 vars in 1138 bytes} [Mon Aug 29 20:21:43 2022] POST /identity/v3/domains => generated 267 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:45.779821 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb26424-3519-4b41-a3f9-c6b8bb2c0154 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:45.780363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb26424-3519-4b41-a3f9-c6b8bb2c0154 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:45.780475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb26424-3519-4b41-a3f9-c6b8bb2c0154 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:45.781747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 610/1170] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:21:45.787258 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366bcf8-e632-4b41-8e9c-8d5d2227422d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:45.787684 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366bcf8-e632-4b41-8e9c-8d5d2227422d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:45.788340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366bcf8-e632-4b41-8e9c-8d5d2227422d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:45.806851 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6366bcf8-e632-4b41-8e9c-8d5d2227422d None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:45.808080 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6366bcf8-e632-4b41-8e9c-8d5d2227422d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:45.873889 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 561/1171] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:46.141480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb26424-3519-4b41-a3f9-c6b8bb2c0154 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:46.239269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb26424-3519-4b41-a3f9-c6b8bb2c0154 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:46.272792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b73a1b-3543-4d77-b13c-1468c54aa430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:46.273402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b73a1b-3543-4d77-b13c-1468c54aa430 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:46.273680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b73a1b-3543-4d77-b13c-1468c54aa430 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:46.273950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b73a1b-3543-4d77-b13c-1468c54aa430 None admin] PATH_INFO: `/v3/domains/f356b35edd00464d9b1e9b7af3753c66` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:46.283787 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b73a1b-3543-4d77-b13c-1468c54aa430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f356b35edd00464d9b1e9b7af3753c66)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:46.285292 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b73a1b-3543-4d77-b13c-1468c54aa430 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:46.287251 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 611/1172] 10.222.0.18 () {56 vars in 1183 bytes} [Mon Aug 29 20:21:46 2022] GET /identity/v3/domains/f356b35edd00464d9b1e9b7af3753c66 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:46.292984 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366bcf8-e632-4b41-8e9c-8d5d2227422d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:46.385852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:46.392539 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366bcf8-e632-4b41-8e9c-8d5d2227422d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:46.427257 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc810b-7cfa-4d18-9108-b3fd7628937c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:46.428384 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc810b-7cfa-4d18-9108-b3fd7628937c None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:46.428665 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc810b-7cfa-4d18-9108-b3fd7628937c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:46.428934 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc810b-7cfa-4d18-9108-b3fd7628937c None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:46.429569 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc810b-7cfa-4d18-9108-b3fd7628937c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:46.430991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc810b-7cfa-4d18-9108-b3fd7628937c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:46.456642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 562/1173] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 29 20:21:46 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:21:48.393815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59eaf4-0388-4f79-be7c-fea0b45d1d3b None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:48.394335 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59eaf4-0388-4f79-be7c-fea0b45d1d3b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:48.394428 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59eaf4-0388-4f79-be7c-fea0b45d1d3b None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:48.395702 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 612/1174] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:48.400948 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1d665-12ec-4e12-91e4-5b11fbe1c978 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:48.401388 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1d665-12ec-4e12-91e4-5b11fbe1c978 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:48.401486 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1d665-12ec-4e12-91e4-5b11fbe1c978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:48.419997 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b6f1d665-12ec-4e12-91e4-5b11fbe1c978 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:48.421179 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b6f1d665-12ec-4e12-91e4-5b11fbe1c978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:48.485574 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 563/1175] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:48.760283 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59eaf4-0388-4f79-be7c-fea0b45d1d3b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:48.786046 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:48.861932 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59eaf4-0388-4f79-be7c-fea0b45d1d3b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:48.894187 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f1f79-aa8e-44ce-9302-1b0307b94e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:48.894814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f1f79-aa8e-44ce-9302-1b0307b94e2f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:48.895098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f1f79-aa8e-44ce-9302-1b0307b94e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:48.895379 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f1f79-aa8e-44ce-9302-1b0307b94e2f None admin] PATH_INFO: `/v3/domains/f356b35edd00464d9b1e9b7af3753c66` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:48.897211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f1f79-aa8e-44ce-9302-1b0307b94e2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=f356b35edd00464d9b1e9b7af3753c66)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:48.898634 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f1f79-aa8e-44ce-9302-1b0307b94e2f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:48.900208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 613/1176] 10.222.0.18 () {56 vars in 1183 bytes} [Mon Aug 29 20:21:48 2022] GET /identity/v3/domains/f356b35edd00464d9b1e9b7af3753c66 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:48.905922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1d665-12ec-4e12-91e4-5b11fbe1c978 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:48.979771 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:49.001605 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1d665-12ec-4e12-91e4-5b11fbe1c978 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:49.034914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc2e51-19ab-4cd1-95d7-986a7ac95277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:49.035764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc2e51-19ab-4cd1-95d7-986a7ac95277 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:49.036081 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc2e51-19ab-4cd1-95d7-986a7ac95277 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:49.036354 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc2e51-19ab-4cd1-95d7-986a7ac95277 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:49.037085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc2e51-19ab-4cd1-95d7-986a7ac95277 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:49.038442 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fc2e51-19ab-4cd1-95d7-986a7ac95277 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:49.059176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 564/1177] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 29 20:21:48 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:21:50.990203 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820b2e4-ad2a-4888-8aa5-06a1355c84f1 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:50.990762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820b2e4-ad2a-4888-8aa5-06a1355c84f1 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:50.990762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820b2e4-ad2a-4888-8aa5-06a1355c84f1 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:50.992145 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 614/1178] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:50.997406 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c23d87-4668-4652-b2aa-4841817a6849 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:50.997862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c23d87-4668-4652-b2aa-4841817a6849 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:50.997966 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c23d87-4668-4652-b2aa-4841817a6849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:51.015923 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d8c23d87-4668-4652-b2aa-4841817a6849 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:51.017135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d8c23d87-4668-4652-b2aa-4841817a6849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:51.085310 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 565/1179] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:51.354054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820b2e4-ad2a-4888-8aa5-06a1355c84f1 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:51.454276 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:51.465927 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820b2e4-ad2a-4888-8aa5-06a1355c84f1 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:51.499282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503dd4f-a70b-465d-a4f8-ac50f0e1be59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:51.499911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503dd4f-a70b-465d-a4f8-ac50f0e1be59 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:51.500217 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503dd4f-a70b-465d-a4f8-ac50f0e1be59 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:51.500490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503dd4f-a70b-465d-a4f8-ac50f0e1be59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:51.510831 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503dd4f-a70b-465d-a4f8-ac50f0e1be59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:21:51.515642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-6503dd4f-a70b-465d-a4f8-ac50f0e1be59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:21:51.515642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:51.515642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:51.515642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:51.515642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:51.515642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:51.515642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:51.515642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:51.515642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:51.515642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:51.515642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:51.515642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:51.515642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:21:51.515642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:21:51.515642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:21:51.515642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:21:51.516764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:21:51.516764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:21:51.516764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:21:51.516764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:21:51.516764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:51.516764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:51.516764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:51.516764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:51.516764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:51.516764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:51.516764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:51.516764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:51.516764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:51.516764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:51.516764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:51.516764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:51.517649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:51.517649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:51.517649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:51.517649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:51.517649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:21:51.517649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:21:51.517649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:21:51.517649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:21:51.517649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:21:51.517649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:21:51.517649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:21:51.517649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:21:51.517649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 615/1180] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:21:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:21:51.523485 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c23d87-4668-4652-b2aa-4841817a6849 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:51.584378 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:51.620282 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c23d87-4668-4652-b2aa-4841817a6849 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:51.653237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51caddee-453c-49af-bf11-eece46bb9dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:51.653843 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51caddee-453c-49af-bf11-eece46bb9dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:51.654123 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51caddee-453c-49af-bf11-eece46bb9dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:51.654391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51caddee-453c-49af-bf11-eece46bb9dc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:51.655047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51caddee-453c-49af-bf11-eece46bb9dc5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:51.655345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51caddee-453c-49af-bf11-eece46bb9dc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:51.656736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51caddee-453c-49af-bf11-eece46bb9dc5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:51.666064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 566/1181] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:21:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:21:51.672053 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503dd4f-a70b-465d-a4f8-ac50f0e1be59 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:51.767773 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503dd4f-a70b-465d-a4f8-ac50f0e1be59 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:51.800761 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb421bb0-4093-448d-96b7-f2a63d62b5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:51.801384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb421bb0-4093-448d-96b7-f2a63d62b5da None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:51.801665 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb421bb0-4093-448d-96b7-f2a63d62b5da None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:51.801937 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb421bb0-4093-448d-96b7-f2a63d62b5da None admin] PATH_INFO: `/v3/users/6fbb7f8354904f88801743727d3371a0` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:51.814600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb421bb0-4093-448d-96b7-f2a63d62b5da None admin] RBAC: Authorizing `identity:get_user(user_id=6fbb7f8354904f88801743727d3371a0)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:51.816047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb421bb0-4093-448d-96b7-f2a63d62b5da None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:51.817609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 616/1182] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:21:51 2022] GET /identity/v3/users/6fbb7f8354904f88801743727d3371a0 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:51.823604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51caddee-453c-49af-bf11-eece46bb9dc5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:51.922808 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51caddee-453c-49af-bf11-eece46bb9dc5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:51.946845 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:51.955601 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39002cc7-4d42-46bc-a5ea-6844932cc8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:51.956225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39002cc7-4d42-46bc-a5ea-6844932cc8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:51.956503 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39002cc7-4d42-46bc-a5ea-6844932cc8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:51.956770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39002cc7-4d42-46bc-a5ea-6844932cc8c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:52.020638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39002cc7-4d42-46bc-a5ea-6844932cc8c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:52.022754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39002cc7-4d42-46bc-a5ea-6844932cc8c5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:52.116036 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 567/1183] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:21:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:21:52.122110 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb421bb0-4093-448d-96b7-f2a63d62b5da None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:52.134692 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:52.219575 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb421bb0-4093-448d-96b7-f2a63d62b5da None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:52.252026 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7ce12-980d-4ecb-9325-b95d770e8b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:52.252648 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7ce12-980d-4ecb-9325-b95d770e8b04 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:52.252929 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7ce12-980d-4ecb-9325-b95d770e8b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:52.253202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7ce12-980d-4ecb-9325-b95d770e8b04 None admin] PATH_INFO: `/v3/projects/61f65da6633d4dba823edb84c414dc15` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:52.254300 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7ce12-980d-4ecb-9325-b95d770e8b04 None admin] RBAC: Authorizing `identity:get_project(project_id=61f65da6633d4dba823edb84c414dc15)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:52.256344 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7ce12-980d-4ecb-9325-b95d770e8b04 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:52.257977 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 617/1184] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:21:52 2022] GET /identity/v3/projects/61f65da6633d4dba823edb84c414dc15 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:52.264154 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39002cc7-4d42-46bc-a5ea-6844932cc8c5 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:52.290324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:52.360635 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39002cc7-4d42-46bc-a5ea-6844932cc8c5 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:52.394585 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94256e13-027c-47f5-a35a-128096c89c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:52.395173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94256e13-027c-47f5-a35a-128096c89c4e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:52.395453 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94256e13-027c-47f5-a35a-128096c89c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:52.395724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94256e13-027c-47f5-a35a-128096c89c4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:52.396499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94256e13-027c-47f5-a35a-128096c89c4e None admin] RBAC: Adding query filter params (role.id=322da7bd64024770a64bdb73e5c54719, scope.project.id=61f65da6633d4dba823edb84c414dc15, user.id=6fbb7f8354904f88801743727d3371a0) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:52.396806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94256e13-027c-47f5-a35a-128096c89c4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:52.398230 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94256e13-027c-47f5-a35a-128096c89c4e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:52.403436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 568/1185] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:21:52 2022] GET /identity/v3/role_assignments?user.id=6fbb7f8354904f88801743727d3371a0&scope.project.id=61f65da6633d4dba823edb84c414dc15&role.id=322da7bd64024770a64bdb73e5c54719 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:54.299270 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1ce31-9753-4b33-8c25-58f177aabe85 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:54.299807 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1ce31-9753-4b33-8c25-58f177aabe85 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:54.299807 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1ce31-9753-4b33-8c25-58f177aabe85 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:54.301211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 618/1186] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:54.307104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4708f-38c1-45bf-85a5-bac28937ca81 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:54.307557 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4708f-38c1-45bf-85a5-bac28937ca81 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:54.307647 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4708f-38c1-45bf-85a5-bac28937ca81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:54.325350 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b9f4708f-38c1-45bf-85a5-bac28937ca81 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:54.326529 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b9f4708f-38c1-45bf-85a5-bac28937ca81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:54.390241 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 569/1187] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:54.659417 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1ce31-9753-4b33-8c25-58f177aabe85 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:54.737816 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:54.760567 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1ce31-9753-4b33-8c25-58f177aabe85 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:54.795381 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d250b64-3962-4d86-86ec-25d5d441b463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:54.796036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d250b64-3962-4d86-86ec-25d5d441b463 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:54.796313 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d250b64-3962-4d86-86ec-25d5d441b463 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:54.796580 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d250b64-3962-4d86-86ec-25d5d441b463 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:54.806324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d250b64-3962-4d86-86ec-25d5d441b463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:21:54.811075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-4d250b64-3962-4d86-86ec-25d5d441b463 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:21:54.811075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:54.811075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:54.811075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:54.811075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:54.811075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:54.811075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:54.811075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:54.811075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:54.811075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:54.811075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:54.811075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:54.811075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:21:54.811075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:21:54.811075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:21:54.811075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:21:54.812226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:21:54.812226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:21:54.812226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:21:54.812226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:21:54.812226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:54.812226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:54.812226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:54.812226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:54.812226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:54.812226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:54.812226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:54.812226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:54.812226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:54.812226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:54.812226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:54.812226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:54.813268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:54.813268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:54.813268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:54.813268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:54.813268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:21:54.813268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:21:54.813268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:21:54.813268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:21:54.813268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:21:54.813268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:21:54.813268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:21:54.813268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:21:54.813268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 619/1188] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:21:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:21:54.818368 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4708f-38c1-45bf-85a5-bac28937ca81 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:54.840208 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:54.919361 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4708f-38c1-45bf-85a5-bac28937ca81 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:54.952483 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d6b034-e57e-4e27-8a00-1a7bdfe8a1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:54.953090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6b034-e57e-4e27-8a00-1a7bdfe8a1be None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:54.953373 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6b034-e57e-4e27-8a00-1a7bdfe8a1be None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:54.953649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d6b034-e57e-4e27-8a00-1a7bdfe8a1be None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:54.954311 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d6b034-e57e-4e27-8a00-1a7bdfe8a1be None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:54.954615 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d6b034-e57e-4e27-8a00-1a7bdfe8a1be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:54.955997 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d6b034-e57e-4e27-8a00-1a7bdfe8a1be None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:54.964476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 570/1189] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:21:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:54.970267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d250b64-3962-4d86-86ec-25d5d441b463 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:55.071186 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d250b64-3962-4d86-86ec-25d5d441b463 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:55.104896 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99255c8b-8584-4d38-8c73-765d7f4538b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:55.105549 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99255c8b-8584-4d38-8c73-765d7f4538b7 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:55.105833 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99255c8b-8584-4d38-8c73-765d7f4538b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:55.106109 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99255c8b-8584-4d38-8c73-765d7f4538b7 None admin] PATH_INFO: `/v3/users/6fbb7f8354904f88801743727d3371a0` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:55.107248 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99255c8b-8584-4d38-8c73-765d7f4538b7 None admin] RBAC: Authorizing `identity:get_user(user_id=6fbb7f8354904f88801743727d3371a0)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:55.108734 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99255c8b-8584-4d38-8c73-765d7f4538b7 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:55.110542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 620/1190] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:21:54 2022] GET /identity/v3/users/6fbb7f8354904f88801743727d3371a0 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:55.116842 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d6b034-e57e-4e27-8a00-1a7bdfe8a1be None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:55.199974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:55.218757 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d6b034-e57e-4e27-8a00-1a7bdfe8a1be None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:55.263056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e4055-731d-4262-a676-9b8fe09f257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:55.264873 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e4055-731d-4262-a676-9b8fe09f257d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:55.265153 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e4055-731d-4262-a676-9b8fe09f257d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:55.265452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e4055-731d-4262-a676-9b8fe09f257d None admin] PATH_INFO: `/v3/projects/61f65da6633d4dba823edb84c414dc15` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:55.266615 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e4055-731d-4262-a676-9b8fe09f257d None admin] RBAC: Authorizing `identity:get_project(project_id=61f65da6633d4dba823edb84c414dc15)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:55.268117 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e4055-731d-4262-a676-9b8fe09f257d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:55.271504 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 571/1191] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:21:55 2022] GET /identity/v3/projects/61f65da6633d4dba823edb84c414dc15 => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:55.277779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99255c8b-8584-4d38-8c73-765d7f4538b7 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:55.289095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:55.382913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99255c8b-8584-4d38-8c73-765d7f4538b7 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:55.417377 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908493f-0b80-4a7a-98b5-45f7e7413f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:55.418043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908493f-0b80-4a7a-98b5-45f7e7413f0a None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:55.418327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908493f-0b80-4a7a-98b5-45f7e7413f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:55.418602 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908493f-0b80-4a7a-98b5-45f7e7413f0a None admin] PATH_INFO: `/v3/projects/61f65da6633d4dba823edb84c414dc15/users/6fbb7f8354904f88801743727d3371a0/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:55.420508 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908493f-0b80-4a7a-98b5-45f7e7413f0a None admin] RBAC: Authorizing `identity:create_grant(project_id=61f65da6633d4dba823edb84c414dc15, user_id=6fbb7f8354904f88801743727d3371a0, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:55.421943 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908493f-0b80-4a7a-98b5-45f7e7413f0a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:55.434532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 621/1192] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:21:55 2022] PUT /identity/v3/projects/61f65da6633d4dba823edb84c414dc15/users/6fbb7f8354904f88801743727d3371a0/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:21:57.346324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e59974-620d-4e1f-9dd1-2685b497399a None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:57.346860 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e59974-620d-4e1f-9dd1-2685b497399a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:57.346860 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e59974-620d-4e1f-9dd1-2685b497399a None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:57.348405 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 572/1193] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:21:57.354039 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8bd71-5b63-45a5-87a5-779e2ef0dc73 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:57.354468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8bd71-5b63-45a5-87a5-779e2ef0dc73 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:57.354597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8bd71-5b63-45a5-87a5-779e2ef0dc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:57.373393 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-48f8bd71-5b63-45a5-87a5-779e2ef0dc73 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:21:57.374591 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-48f8bd71-5b63-45a5-87a5-779e2ef0dc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:21:57.439599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:57.448122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 622/1194] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:21:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:21:57.719851 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e59974-620d-4e1f-9dd1-2685b497399a None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:57.820144 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e59974-620d-4e1f-9dd1-2685b497399a None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:57.852486 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71a5bf-d774-45d0-9fae-8aec42d6a8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:57.853090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71a5bf-d774-45d0-9fae-8aec42d6a8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:57.853364 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71a5bf-d774-45d0-9fae-8aec42d6a8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:57.853633 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71a5bf-d774-45d0-9fae-8aec42d6a8e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:57.863598 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71a5bf-d774-45d0-9fae-8aec42d6a8e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:21:57.868357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-5f71a5bf-d774-45d0-9fae-8aec42d6a8e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:21:57.868357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:21:57.868357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:21:57.868357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:21:57.868357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:21:57.868357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:21:57.868357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:21:57.868357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:21:57.868357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:21:57.868357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:21:57.868357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:21:57.868357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:21:57.868357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:21:57.868357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:21:57.868357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:21:57.868357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:21:57.869562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:21:57.869562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:21:57.869562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:21:57.869562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:21:57.869562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:21:57.869562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:21:57.869562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:21:57.869562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:21:57.869562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:21:57.869562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:21:57.869562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:21:57.869562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:21:57.869562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:21:57.869562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:21:57.869562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:21:57.869562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:21:57.870649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:21:57.870649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:21:57.870649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:21:57.870649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:21:57.870649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:21:57.870649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:21:57.870649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:21:57.870649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:21:57.870649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:21:57.870649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:21:57.870649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:21:57.870649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:21:57.870649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 573/1195] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:21:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:21:57.875555 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8bd71-5b63-45a5-87a5-779e2ef0dc73 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:57.975488 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8bd71-5b63-45a5-87a5-779e2ef0dc73 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:58.008586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d3a73-ebad-4cd9-a2ef-354b7b477d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:58.009185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d3a73-ebad-4cd9-a2ef-354b7b477d9e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:58.009466 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d3a73-ebad-4cd9-a2ef-354b7b477d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:58.009743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d3a73-ebad-4cd9-a2ef-354b7b477d9e None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:58.010405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d3a73-ebad-4cd9-a2ef-354b7b477d9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:58.010708 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d3a73-ebad-4cd9-a2ef-354b7b477d9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:58.012082 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d3a73-ebad-4cd9-a2ef-354b7b477d9e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:58.019773 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 623/1196] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:21:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:58.026366 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71a5bf-d774-45d0-9fae-8aec42d6a8e0 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:58.039588 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:58.125812 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71a5bf-d774-45d0-9fae-8aec42d6a8e0 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:58.157522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6777d5-44e3-4510-b727-9381c2987f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:58.158134 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6777d5-44e3-4510-b727-9381c2987f58 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:58.158410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6777d5-44e3-4510-b727-9381c2987f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:58.158680 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6777d5-44e3-4510-b727-9381c2987f58 None admin] PATH_INFO: `/v3/users/6fbb7f8354904f88801743727d3371a0` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:58.159774 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6777d5-44e3-4510-b727-9381c2987f58 None admin] RBAC: Authorizing `identity:get_user(user_id=6fbb7f8354904f88801743727d3371a0)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:58.162006 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6777d5-44e3-4510-b727-9381c2987f58 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:58.163556 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 574/1197] 10.222.0.18 () {56 vars in 1177 bytes} [Mon Aug 29 20:21:58 2022] GET /identity/v3/users/6fbb7f8354904f88801743727d3371a0 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:58.169571 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d3a73-ebad-4cd9-a2ef-354b7b477d9e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:58.226208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:58.275089 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d3a73-ebad-4cd9-a2ef-354b7b477d9e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:58.308348 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019a518-225a-4eec-b042-9a1cfa94336e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:58.308956 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019a518-225a-4eec-b042-9a1cfa94336e None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:58.309233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019a518-225a-4eec-b042-9a1cfa94336e None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:58.309501 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019a518-225a-4eec-b042-9a1cfa94336e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:58.373327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019a518-225a-4eec-b042-9a1cfa94336e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:58.374752 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019a518-225a-4eec-b042-9a1cfa94336e None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:58.429041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:58.469338 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 624/1198] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:21:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:21:58.476064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6777d5-44e3-4510-b727-9381c2987f58 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:58.556779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:21:58.578398 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6777d5-44e3-4510-b727-9381c2987f58 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:58.612192 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13ebbe-9a92-468c-a0cc-f367b2649fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:58.612822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13ebbe-9a92-468c-a0cc-f367b2649fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:58.613103 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13ebbe-9a92-468c-a0cc-f367b2649fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:58.613376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13ebbe-9a92-468c-a0cc-f367b2649fd3 None admin] PATH_INFO: `/v3/projects/61f65da6633d4dba823edb84c414dc15` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:58.614502 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f13ebbe-9a92-468c-a0cc-f367b2649fd3 None admin] RBAC: Authorizing `identity:get_project(project_id=61f65da6633d4dba823edb84c414dc15)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:58.615960 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f13ebbe-9a92-468c-a0cc-f367b2649fd3 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:58.617657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 575/1199] 10.222.0.18 () {56 vars in 1186 bytes} [Mon Aug 29 20:21:58 2022] GET /identity/v3/projects/61f65da6633d4dba823edb84c414dc15 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:21:58.623749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019a518-225a-4eec-b042-9a1cfa94336e None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:21:58.719414 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019a518-225a-4eec-b042-9a1cfa94336e None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:21:58.751860 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25e32f-014a-4269-8a67-078489c73d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:21:58.752485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25e32f-014a-4269-8a67-078489c73d20 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:21:58.752760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25e32f-014a-4269-8a67-078489c73d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:21:58.753028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25e32f-014a-4269-8a67-078489c73d20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:21:58.753741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25e32f-014a-4269-8a67-078489c73d20 None admin] RBAC: Adding query filter params (role.id=322da7bd64024770a64bdb73e5c54719, scope.project.id=61f65da6633d4dba823edb84c414dc15, user.id=6fbb7f8354904f88801743727d3371a0) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:21:58.754046 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25e32f-014a-4269-8a67-078489c73d20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:21:58.756015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25e32f-014a-4269-8a67-078489c73d20 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:21:58.761572 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 625/1200] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:21:58 2022] GET /identity/v3/role_assignments?user.id=6fbb7f8354904f88801743727d3371a0&scope.project.id=61f65da6633d4dba823edb84c414dc15&role.id=322da7bd64024770a64bdb73e5c54719 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:22:08.608089 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=27 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:08.759098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:54.750178 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667b5b5-05e7-4672-a1db-02f59e0407c8 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:54.750780 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667b5b5-05e7-4672-a1db-02f59e0407c8 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:54.750780 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667b5b5-05e7-4672-a1db-02f59e0407c8 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:54.752607 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 576/1201] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:22:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:22:54.758512 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d25299-f301-46fb-b676-d38bf88d4d1a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:54.759007 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d25299-f301-46fb-b676-d38bf88d4d1a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:54.759183 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d25299-f301-46fb-b676-d38bf88d4d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:54.781629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-c1d25299-f301-46fb-b676-d38bf88d4d1a None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:54.782886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c1d25299-f301-46fb-b676-d38bf88d4d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:54.896045 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 626/1202] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:22:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:22:55.166490 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667b5b5-05e7-4672-a1db-02f59e0407c8 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:55.337925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667b5b5-05e7-4672-a1db-02f59e0407c8 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:55.389921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47579e02-1626-464c-a990-6a7ea46e92bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:55.390634 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47579e02-1626-464c-a990-6a7ea46e92bc None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:55.390914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47579e02-1626-464c-a990-6a7ea46e92bc None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:55.391240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47579e02-1626-464c-a990-6a7ea46e92bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:55.405763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47579e02-1626-464c-a990-6a7ea46e92bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:55.407360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47579e02-1626-464c-a990-6a7ea46e92bc None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:55.425879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-47579e02-1626-464c-a990-6a7ea46e92bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:22:55.425879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:22:55.425879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:22:55.425879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:22:55.425879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:22:55.425879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:22:55.425879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:22:55.425879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:22:55.425879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:22:55.425879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:22:55.425879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:22:55.425879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:22:55.425879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:22:55.425879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:22:55.425879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:22:55.425879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:22:55.427000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:22:55.427000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:22:55.427000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:22:55.427000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:22:55.427000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:22:55.427000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:22:55.427000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:22:55.427000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:22:55.427000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:22:55.427000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:22:55.427000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:22:55.427000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:22:55.427000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:22:55.427000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:22:55.427000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:22:55.427000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:22:55.427000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:22:55.428367 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:22:55.428367 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:22:55.428367 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:22:55.428367 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 577/1203] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:22:55.435976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d25299-f301-46fb-b676-d38bf88d4d1a None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:55.619105 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d25299-f301-46fb-b676-d38bf88d4d1a None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:55.664515 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:55.681922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b927f35a-2a90-44b9-bbcc-c4bdd515093c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:55.682532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927f35a-2a90-44b9-bbcc-c4bdd515093c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:55.682819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927f35a-2a90-44b9-bbcc-c4bdd515093c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:55.683162 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927f35a-2a90-44b9-bbcc-c4bdd515093c None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:55.683856 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b927f35a-2a90-44b9-bbcc-c4bdd515093c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:22:55.684249 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b927f35a-2a90-44b9-bbcc-c4bdd515093c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:55.685894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b927f35a-2a90-44b9-bbcc-c4bdd515093c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:55.703994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 627/1204] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:22:55.708563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47579e02-1626-464c-a990-6a7ea46e92bc None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:55.785033 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:55.841542 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47579e02-1626-464c-a990-6a7ea46e92bc None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:55.886619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78a7e8-08fb-43fb-9f57-bb8b954dc2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:55.887274 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78a7e8-08fb-43fb-9f57-bb8b954dc2ff None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:55.887546 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78a7e8-08fb-43fb-9f57-bb8b954dc2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:55.887909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78a7e8-08fb-43fb-9f57-bb8b954dc2ff None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:55.888747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78a7e8-08fb-43fb-9f57-bb8b954dc2ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:55.890244 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78a7e8-08fb-43fb-9f57-bb8b954dc2ff None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:55.918019 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 578/1205] 10.222.0.18 () {60 vars in 1132 bytes} [Mon Aug 29 20:22:55 2022] POST /identity/v3/users => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:22:58.060626 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37794f7f-ed93-495b-bdf9-a4bef229029d None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:58.061410 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37794f7f-ed93-495b-bdf9-a4bef229029d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:58.061410 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37794f7f-ed93-495b-bdf9-a4bef229029d None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:58.062708 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 628/1206] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:22:58.068128 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82d004-a483-4dc9-8c83-75b21ce1151d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:58.068650 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82d004-a483-4dc9-8c83-75b21ce1151d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:58.068771 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82d004-a483-4dc9-8c83-75b21ce1151d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:58.093588 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-3d82d004-a483-4dc9-8c83-75b21ce1151d None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:22:58.094839 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-3d82d004-a483-4dc9-8c83-75b21ce1151d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:22:58.180452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 579/1207] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:22:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:22:58.478271 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37794f7f-ed93-495b-bdf9-a4bef229029d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:58.605496 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37794f7f-ed93-495b-bdf9-a4bef229029d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:58.644408 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf7553-19f5-4b01-b94b-75e22c08a762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:58.645057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf7553-19f5-4b01-b94b-75e22c08a762 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:58.645344 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf7553-19f5-4b01-b94b-75e22c08a762 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:58.645622 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf7553-19f5-4b01-b94b-75e22c08a762 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:58.659165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf7553-19f5-4b01-b94b-75e22c08a762 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:22:58.664177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-abaf7553-19f5-4b01-b94b-75e22c08a762 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:22:58.664177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:22:58.664177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:22:58.664177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:22:58.664177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:22:58.664177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:22:58.664177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:22:58.664177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:22:58.664177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:22:58.664177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:22:58.664177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:22:58.664177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:22:58.664177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:22:58.664177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:22:58.664177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:22:58.664177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:22:58.670672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:22:58.670672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:22:58.670672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:22:58.670672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:22:58.670672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:22:58.670672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:22:58.670672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:22:58.670672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:22:58.670672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:22:58.670672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:22:58.670672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:22:58.670672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:22:58.670672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:22:58.670672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:22:58.670672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:22:58.670672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:22:58.672360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:22:58.672360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:22:58.672360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:22:58.672360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:22:58.672360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:22:58.672360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:22:58.672360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:22:58.672360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:22:58.672360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:22:58.672360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:22:58.672360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:22:58.672360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:22:58.672360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 629/1208] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:22:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:22:58.673273 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82d004-a483-4dc9-8c83-75b21ce1151d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:58.722727 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:58.789948 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82d004-a483-4dc9-8c83-75b21ce1151d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:58.827648 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb34cf0b-bb97-4a9e-9a07-14183bf9aa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:58.828324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34cf0b-bb97-4a9e-9a07-14183bf9aa93 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:58.828634 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34cf0b-bb97-4a9e-9a07-14183bf9aa93 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:58.828899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34cf0b-bb97-4a9e-9a07-14183bf9aa93 None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:58.829613 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb34cf0b-bb97-4a9e-9a07-14183bf9aa93 None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:22:58.829919 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb34cf0b-bb97-4a9e-9a07-14183bf9aa93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:58.831401 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb34cf0b-bb97-4a9e-9a07-14183bf9aa93 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:58.840504 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 580/1209] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:22:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:22:58.846906 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf7553-19f5-4b01-b94b-75e22c08a762 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:58.909300 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:58.970119 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf7553-19f5-4b01-b94b-75e22c08a762 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:59.019056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18f115-1946-4123-895b-4b09c1449f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:59.020377 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18f115-1946-4123-895b-4b09c1449f50 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:59.020377 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18f115-1946-4123-895b-4b09c1449f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:59.020591 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18f115-1946-4123-895b-4b09c1449f50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:59.036732 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18f115-1946-4123-895b-4b09c1449f50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:59.038260 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18f115-1946-4123-895b-4b09c1449f50 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:59.055855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-aa18f115-1946-4123-895b-4b09c1449f50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:22:59.055855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:22:59.055855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:22:59.055855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:22:59.055855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:22:59.055855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:22:59.055855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:22:59.055855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:22:59.055855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:22:59.055855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:22:59.055855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:22:59.055855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:22:59.055855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:22:59.055855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:22:59.055855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:22:59.055855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:22:59.057062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:22:59.057062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:22:59.057062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:22:59.057062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:22:59.057062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:22:59.057062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:22:59.057062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:22:59.057062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:22:59.057062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:22:59.057062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:22:59.057062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:22:59.057062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:22:59.057062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:22:59.057062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:22:59.057062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:22:59.057062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:22:59.057062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:22:59.058125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:22:59.058125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:22:59.058125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:22:59.058125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 630/1210] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:22:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:22:59.063827 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb34cf0b-bb97-4a9e-9a07-14183bf9aa93 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:59.181606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:59.185479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb34cf0b-bb97-4a9e-9a07-14183bf9aa93 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:59.225095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373448a-9f41-40da-ae19-854eb4cab3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:59.225736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373448a-9f41-40da-ae19-854eb4cab3ed None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:59.226063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373448a-9f41-40da-ae19-854eb4cab3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:59.226333 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373448a-9f41-40da-ae19-854eb4cab3ed None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:59.227059 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373448a-9f41-40da-ae19-854eb4cab3ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:22:59.227346 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373448a-9f41-40da-ae19-854eb4cab3ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:59.228865 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373448a-9f41-40da-ae19-854eb4cab3ed None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:59.242098 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 581/1211] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:22:59.248366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18f115-1946-4123-895b-4b09c1449f50 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:59.368965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18f115-1946-4123-895b-4b09c1449f50 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:59.370481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:59.409531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cbf46d-6331-48ce-a48f-ddf61e9f7883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:59.410241 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbf46d-6331-48ce-a48f-ddf61e9f7883 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:59.410571 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbf46d-6331-48ce-a48f-ddf61e9f7883 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:59.410893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbf46d-6331-48ce-a48f-ddf61e9f7883 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:59.420260 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cbf46d-6331-48ce-a48f-ddf61e9f7883 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:59.421732 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cbf46d-6331-48ce-a48f-ddf61e9f7883 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:59.438389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-29cbf46d-6331-48ce-a48f-ddf61e9f7883 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:22:59.438389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:22:59.438389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:22:59.438389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:22:59.438389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:22:59.438389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:22:59.438389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:22:59.438389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:22:59.438389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:22:59.438389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:22:59.438389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:22:59.438389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:22:59.438389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:22:59.438389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:22:59.438389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:22:59.438389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:22:59.439653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:22:59.439653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:22:59.439653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:22:59.439653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:22:59.439653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:22:59.439653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:22:59.439653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:22:59.439653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:22:59.439653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:22:59.439653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:22:59.439653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:22:59.439653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:22:59.439653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:22:59.439653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:22:59.439653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:22:59.439653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:22:59.439653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:22:59.439653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:22:59.440779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:22:59.440779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:22:59.440779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:22:59.440779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:22:59.440779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:22:59.440779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:22:59.440779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:22:59.440779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:22:59.440779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:22:59.440779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:22:59.440779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 631/1212] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:22:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:22:59.445726 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373448a-9f41-40da-ae19-854eb4cab3ed None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:59.570716 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373448a-9f41-40da-ae19-854eb4cab3ed None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:59.610412 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a94fb7-9ed2-48db-8e06-bc4b898f41e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:59.611132 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a94fb7-9ed2-48db-8e06-bc4b898f41e2 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:59.611418 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a94fb7-9ed2-48db-8e06-bc4b898f41e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:59.611694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a94fb7-9ed2-48db-8e06-bc4b898f41e2 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:59.612466 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a94fb7-9ed2-48db-8e06-bc4b898f41e2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:22:59.612774 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a94fb7-9ed2-48db-8e06-bc4b898f41e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:59.614208 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a94fb7-9ed2-48db-8e06-bc4b898f41e2 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:22:59.640538 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 582/1213] 10.222.0.18 () {56 vars in 1143 bytes} [Mon Aug 29 20:22:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:22:59.646359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cbf46d-6331-48ce-a48f-ddf61e9f7883 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:22:59.767899 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cbf46d-6331-48ce-a48f-ddf61e9f7883 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:22:59.807494 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42ed64-8cf3-4edd-a9a4-75f7ba3a8b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:22:59.808862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42ed64-8cf3-4edd-a9a4-75f7ba3a8b9c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:22:59.808862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42ed64-8cf3-4edd-a9a4-75f7ba3a8b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:22:59.809082 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42ed64-8cf3-4edd-a9a4-75f7ba3a8b9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:22:59.851171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:22:59.905329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f42ed64-8cf3-4edd-a9a4-75f7ba3a8b9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:22:59.907268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f42ed64-8cf3-4edd-a9a4-75f7ba3a8b9c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:00.066014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 632/1214] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:22:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:23:00.067091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a94fb7-9ed2-48db-8e06-bc4b898f41e2 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:00.088710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:00.206543 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a94fb7-9ed2-48db-8e06-bc4b898f41e2 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:00.250533 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14103b73-f605-4890-b733-cc6f05bc5b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:00.251436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14103b73-f605-4890-b733-cc6f05bc5b9e None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:00.251738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14103b73-f605-4890-b733-cc6f05bc5b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:00.252119 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14103b73-f605-4890-b733-cc6f05bc5b9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:00.265883 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14103b73-f605-4890-b733-cc6f05bc5b9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:00.267407 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14103b73-f605-4890-b733-cc6f05bc5b9e None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:00.287294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-14103b73-f605-4890-b733-cc6f05bc5b9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:00.287294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:00.287294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:00.287294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:00.287294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:00.287294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:00.287294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:00.287294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:00.287294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:00.287294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:00.287294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:00.287294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:00.287294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:23:00.287294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:23:00.287294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:23:00.287294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:23:00.288375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:00.288375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:00.288375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:00.288375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:00.288375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:00.288375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:00.288375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:00.288375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:00.288375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:00.288375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:00.288375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:00.288375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:00.288375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:00.288375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:00.288375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:00.288375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:00.288375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:23:00.289280 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:23:00.289280 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:00.289280 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:23:00.296447 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 583/1215] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:23:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:23:00.296679 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42ed64-8cf3-4edd-a9a4-75f7ba3a8b9c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:00.381914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:00.433252 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42ed64-8cf3-4edd-a9a4-75f7ba3a8b9c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:00.482907 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f37cb-a2d3-4321-a071-64255f5e8941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:00.484160 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f37cb-a2d3-4321-a071-64255f5e8941 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:00.484565 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f37cb-a2d3-4321-a071-64255f5e8941 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:00.485078 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f37cb-a2d3-4321-a071-64255f5e8941 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:00.486098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f37cb-a2d3-4321-a071-64255f5e8941 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:00.486782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f37cb-a2d3-4321-a071-64255f5e8941 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:00.489418 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f37cb-a2d3-4321-a071-64255f5e8941 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:00.505266 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 633/1216] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:00.512742 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14103b73-f605-4890-b733-cc6f05bc5b9e None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:00.606252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:00.642440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14103b73-f605-4890-b733-cc6f05bc5b9e None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:00.687082 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93fc42-d80a-42f0-a64e-391d88d7be15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:00.687746 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93fc42-d80a-42f0-a64e-391d88d7be15 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:00.688092 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93fc42-d80a-42f0-a64e-391d88d7be15 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:00.688446 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93fc42-d80a-42f0-a64e-391d88d7be15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:00.697292 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93fc42-d80a-42f0-a64e-391d88d7be15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:00.699095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93fc42-d80a-42f0-a64e-391d88d7be15 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:00.712512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-ff93fc42-d80a-42f0-a64e-391d88d7be15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:23:00.712512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:00.712512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:00.712512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:00.712512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:00.712512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:00.712512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:00.712512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:00.712512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:00.712512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:00.712512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:00.712512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:00.712512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:23:00.712512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:23:00.712512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:23:00.713642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:23:00.713642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:00.713642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:00.713642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:00.713642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:00.713642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:00.713642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:00.713642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:00.713642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:00.713642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:00.713642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:00.713642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:00.713642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:00.713642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:00.713642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:00.713642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:00.713642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:00.717474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:23:00.717474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:23:00.717474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:23:00.717474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:23:00.717474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:23:00.717474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:23:00.717474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:23:00.717474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:23:00.717474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 584/1217] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:23:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:00.720764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f37cb-a2d3-4321-a071-64255f5e8941 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:00.851847 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f37cb-a2d3-4321-a071-64255f5e8941 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:00.883739 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:00.902876 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d360b97-0733-4f89-b783-2a45ab06e9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:00.904255 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d360b97-0733-4f89-b783-2a45ab06e9bf None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:00.905064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d360b97-0733-4f89-b783-2a45ab06e9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:00.905470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d360b97-0733-4f89-b783-2a45ab06e9bf None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:00.906395 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d360b97-0733-4f89-b783-2a45ab06e9bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:00.906911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d360b97-0733-4f89-b783-2a45ab06e9bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:00.909028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d360b97-0733-4f89-b783-2a45ab06e9bf None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:00.934184 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 634/1218] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:23:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:00.935749 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93fc42-d80a-42f0-a64e-391d88d7be15 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:01.066265 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93fc42-d80a-42f0-a64e-391d88d7be15 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:01.098217 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:01.105833 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946edbd3-ed65-44c3-88b6-d8396c2182f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:01.106383 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946edbd3-ed65-44c3-88b6-d8396c2182f6 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:01.106686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946edbd3-ed65-44c3-88b6-d8396c2182f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:01.107513 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946edbd3-ed65-44c3-88b6-d8396c2182f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:01.108416 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946edbd3-ed65-44c3-88b6-d8396c2182f6 None admin] RBAC: Adding query filter params (role.id=04bc48cc5b6842029b93a845d6744985, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=fa0e659eee5e4a9da8d3d7337cb73798) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:01.108724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946edbd3-ed65-44c3-88b6-d8396c2182f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:01.110269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946edbd3-ed65-44c3-88b6-d8396c2182f6 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:01.116627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 585/1219] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:23:00 2022] GET /identity/v3/role_assignments?user.id=fa0e659eee5e4a9da8d3d7337cb73798&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=04bc48cc5b6842029b93a845d6744985 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:03.401387 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b5713-701c-4a5d-b9e4-555762572c88 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:03.401910 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b5713-701c-4a5d-b9e4-555762572c88 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:03.402088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b5713-701c-4a5d-b9e4-555762572c88 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:03.404227 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 635/1220] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:23:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:23:03.414848 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe9512-814f-412a-a0bf-6e8a7073dbd1 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:03.415375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe9512-814f-412a-a0bf-6e8a7073dbd1 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:03.415602 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe9512-814f-412a-a0bf-6e8a7073dbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:03.444593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-7cfe9512-814f-412a-a0bf-6e8a7073dbd1 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:23:03.446400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7cfe9512-814f-412a-a0bf-6e8a7073dbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:03.527130 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 586/1221] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:23:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:23:03.802353 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b5713-701c-4a5d-b9e4-555762572c88 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:03.911176 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b5713-701c-4a5d-b9e4-555762572c88 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:03.951872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eed9a1-0059-4745-9ba8-f97da14a017b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:03.952535 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eed9a1-0059-4745-9ba8-f97da14a017b None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:03.952818 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eed9a1-0059-4745-9ba8-f97da14a017b None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:03.953093 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eed9a1-0059-4745-9ba8-f97da14a017b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:03.975714 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eed9a1-0059-4745-9ba8-f97da14a017b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:23:03.980862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-b7eed9a1-0059-4745-9ba8-f97da14a017b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:23:03.980862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:03.980862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:03.980862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:03.980862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:03.980862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:03.980862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:03.980862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:03.980862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:03.980862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:03.980862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:03.980862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:03.980862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:23:03.980862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:23:03.980862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:23:03.980862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:23:03.982597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:23:03.982597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:23:03.982597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:23:03.982597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:23:03.982597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:03.982597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:03.982597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:03.982597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:03.982597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:03.982597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:03.982597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:03.982597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:03.982597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:03.982597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:03.982597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:03.982597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:03.983859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:03.983859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:03.983859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:03.983859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:03.983859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:23:03.983859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:23:03.983859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:23:03.983859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:23:03.983859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:23:03.983859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:23:03.983859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:23:03.983859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:03.983859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 636/1222] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:23:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:03.988773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe9512-814f-412a-a0bf-6e8a7073dbd1 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:04.093998 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe9512-814f-412a-a0bf-6e8a7073dbd1 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:04.120022 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:04.134718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33d648-76c9-42f0-bc7f-6d2a70cf9d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:04.135580 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33d648-76c9-42f0-bc7f-6d2a70cf9d8c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:04.135860 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33d648-76c9-42f0-bc7f-6d2a70cf9d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:04.136168 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33d648-76c9-42f0-bc7f-6d2a70cf9d8c None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:04.136842 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33d648-76c9-42f0-bc7f-6d2a70cf9d8c None admin] RBAC: Adding query filter params (name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:04.137148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33d648-76c9-42f0-bc7f-6d2a70cf9d8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:04.138615 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e33d648-76c9-42f0-bc7f-6d2a70cf9d8c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:04.149476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 587/1223] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:23:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:04.156707 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eed9a1-0059-4745-9ba8-f97da14a017b None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:04.209380 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:04.274464 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eed9a1-0059-4745-9ba8-f97da14a017b None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:04.315565 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e95d1b-cb0e-4736-a8e0-1af127bec826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:04.316229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e95d1b-cb0e-4736-a8e0-1af127bec826 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:04.316508 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e95d1b-cb0e-4736-a8e0-1af127bec826 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:04.316780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e95d1b-cb0e-4736-a8e0-1af127bec826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:04.328237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e95d1b-cb0e-4736-a8e0-1af127bec826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:04.329630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e95d1b-cb0e-4736-a8e0-1af127bec826 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:04.345998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-28e95d1b-cb0e-4736-a8e0-1af127bec826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:04.345998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:04.345998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:04.345998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:04.345998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:04.345998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:04.345998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:04.345998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:04.345998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:04.345998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:04.345998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:04.345998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:04.345998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:23:04.345998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:23:04.345998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:23:04.345998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:23:04.347396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:04.347396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:04.347396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:04.347396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:04.347396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:04.347396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:04.347396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:04.347396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:04.347396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:04.347396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:04.347396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:04.347396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:04.347396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:04.347396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:04.347396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:04.347396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:04.347396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:23:04.348668 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:23:04.348668 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:04.348668 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:04.348668 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 637/1224] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:23:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:04.352905 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33d648-76c9-42f0-bc7f-6d2a70cf9d8c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:04.458203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e33d648-76c9-42f0-bc7f-6d2a70cf9d8c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:04.494569 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9fa51-3d70-4fed-b736-f32ec986f1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:04.495245 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9fa51-3d70-4fed-b736-f32ec986f1cc None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:04.495506 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9fa51-3d70-4fed-b736-f32ec986f1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:04.495782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9fa51-3d70-4fed-b736-f32ec986f1cc None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:04.496436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9fa51-3d70-4fed-b736-f32ec986f1cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:04.496733 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9fa51-3d70-4fed-b736-f32ec986f1cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:04.498739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9fa51-3d70-4fed-b736-f32ec986f1cc None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:04.511197 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 588/1225] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:04.517240 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e95d1b-cb0e-4736-a8e0-1af127bec826 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:04.626762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:04.638421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e95d1b-cb0e-4736-a8e0-1af127bec826 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:04.680882 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321dc050-3d2a-4b52-98a8-4b76f1564f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:04.681639 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dc050-3d2a-4b52-98a8-4b76f1564f38 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:04.682007 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dc050-3d2a-4b52-98a8-4b76f1564f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:04.682412 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dc050-3d2a-4b52-98a8-4b76f1564f38 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:04.691835 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321dc050-3d2a-4b52-98a8-4b76f1564f38 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:04.694026 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321dc050-3d2a-4b52-98a8-4b76f1564f38 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:04.706994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-321dc050-3d2a-4b52-98a8-4b76f1564f38 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:23:04.706994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:04.706994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:04.706994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:04.706994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:04.706994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:04.706994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:04.706994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:04.706994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:04.706994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:04.706994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:04.706994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:04.706994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:23:04.706994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:23:04.706994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:23:04.706994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:23:04.708204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:04.708204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:04.708204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:23:04.708204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:23:04.708204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:23:04.708204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:23:04.708204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:04.708204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:04.708204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:04.708204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:04.708204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:04.708204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:04.708204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:04.708204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:04.708204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:04.708204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:04.708204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:04.708204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:04.714469 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:04.714469 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:04.714469 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:23:04.714469 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:23:04.714469 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:23:04.714469 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:23:04.714469 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:23:04.714469 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:23:04.714469 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:23:04.714469 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:04.714469 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 638/1226] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:23:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:04.715124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9fa51-3d70-4fed-b736-f32ec986f1cc None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:04.763884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:04.826991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9fa51-3d70-4fed-b736-f32ec986f1cc None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:04.867339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5477d-6122-4170-a8bc-d4e693652420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:04.867977 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5477d-6122-4170-a8bc-d4e693652420 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:04.868259 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5477d-6122-4170-a8bc-d4e693652420 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:04.868531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5477d-6122-4170-a8bc-d4e693652420 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:04.869290 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a5477d-6122-4170-a8bc-d4e693652420 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:04.869593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a5477d-6122-4170-a8bc-d4e693652420 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:04.871057 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a5477d-6122-4170-a8bc-d4e693652420 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:04.891471 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 589/1227] 10.222.0.18 () {56 vars in 1143 bytes} [Mon Aug 29 20:23:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:04.897120 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321dc050-3d2a-4b52-98a8-4b76f1564f38 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:05.015752 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321dc050-3d2a-4b52-98a8-4b76f1564f38 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:05.055633 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32009da2-9b4d-4c81-9cf4-03b924c10668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:05.056512 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32009da2-9b4d-4c81-9cf4-03b924c10668 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:05.056795 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32009da2-9b4d-4c81-9cf4-03b924c10668 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:05.057130 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32009da2-9b4d-4c81-9cf4-03b924c10668 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:05.070671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32009da2-9b4d-4c81-9cf4-03b924c10668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:05.072196 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32009da2-9b4d-4c81-9cf4-03b924c10668 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:05.086688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-32009da2-9b4d-4c81-9cf4-03b924c10668 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:05.086688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:05.086688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:05.086688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:05.086688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:05.086688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:05.086688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:05.086688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:05.086688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:05.086688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:05.086688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:05.086688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:05.086688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:23:05.086688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:23:05.086688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:23:05.086688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:23:05.088005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:05.088005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:05.088005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:05.088005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:05.088005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:05.088005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:05.088005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:05.088005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:05.088005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:05.088005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:05.088005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:05.088005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:05.088005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:05.088005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:05.088005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:05.088005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:05.088005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:23:05.089069 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:23:05.089069 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:05.089069 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:05.089069 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 639/1228] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:23:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:23:05.093890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5477d-6122-4170-a8bc-d4e693652420 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:05.206086 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5477d-6122-4170-a8bc-d4e693652420 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:05.209130 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:05.259954 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eadd76-84d6-4dce-b912-986407e19174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:05.260660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eadd76-84d6-4dce-b912-986407e19174 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:05.260941 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eadd76-84d6-4dce-b912-986407e19174 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:05.261219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eadd76-84d6-4dce-b912-986407e19174 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:05.261864 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eadd76-84d6-4dce-b912-986407e19174 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:05.262229 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eadd76-84d6-4dce-b912-986407e19174 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:05.263640 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eadd76-84d6-4dce-b912-986407e19174 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:05.276939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 590/1229] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:05.283248 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32009da2-9b4d-4c81-9cf4-03b924c10668 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:05.295766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:05.408627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32009da2-9b4d-4c81-9cf4-03b924c10668 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:05.444413 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4147a3b-6227-4777-8b8c-9799fb228865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:05.445049 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4147a3b-6227-4777-8b8c-9799fb228865 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:05.445325 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4147a3b-6227-4777-8b8c-9799fb228865 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:05.445594 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4147a3b-6227-4777-8b8c-9799fb228865 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:05.452670 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4147a3b-6227-4777-8b8c-9799fb228865 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:05.454096 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4147a3b-6227-4777-8b8c-9799fb228865 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:05.465704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-c4147a3b-6227-4777-8b8c-9799fb228865 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:23:05.465704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:05.465704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:05.465704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:05.465704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:05.465704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:05.465704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:05.465704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:05.465704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:05.465704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:05.465704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:05.465704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:05.465704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:23:05.465704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:23:05.465704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:23:05.466671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:23:05.466671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:05.466671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:05.466671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:05.466671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:05.466671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:05.466671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:05.466671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:05.466671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:05.466671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:05.466671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:05.466671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:05.466671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:05.466671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:05.466671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:05.466671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:05.466671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:05.468869 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:23:05.468869 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:23:05.468869 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:23:05.468869 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:23:05.468869 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:23:05.468869 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:23:05.468869 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:23:05.468869 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:05.468869 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 640/1230] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:23:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:05.474001 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eadd76-84d6-4dce-b912-986407e19174 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:05.593372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eadd76-84d6-4dce-b912-986407e19174 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:05.633707 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3095f-6511-443a-9951-a413d8450d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:05.634313 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3095f-6511-443a-9951-a413d8450d8d None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:05.634595 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3095f-6511-443a-9951-a413d8450d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:05.634866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3095f-6511-443a-9951-a413d8450d8d None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:05.635594 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3095f-6511-443a-9951-a413d8450d8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:05.635900 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3095f-6511-443a-9951-a413d8450d8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:05.637375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3095f-6511-443a-9951-a413d8450d8d None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:05.651405 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 591/1231] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:23:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:23:05.658765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4147a3b-6227-4777-8b8c-9799fb228865 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:05.744256 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:05.785335 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4147a3b-6227-4777-8b8c-9799fb228865 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:05.825619 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911eb6c-eb4b-4898-981c-f5e61171a7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:05.826598 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911eb6c-eb4b-4898-981c-f5e61171a7d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:05.826878 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911eb6c-eb4b-4898-981c-f5e61171a7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:05.827171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911eb6c-eb4b-4898-981c-f5e61171a7d9 None admin] PATH_INFO: `/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/fa0e659eee5e4a9da8d3d7337cb73798/roles/04bc48cc5b6842029b93a845d6744985` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:05.843127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911eb6c-eb4b-4898-981c-f5e61171a7d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=90b25dc95d5a4a068994bdd801d6b8ea, user_id=fa0e659eee5e4a9da8d3d7337cb73798, role_id=04bc48cc5b6842029b93a845d6744985)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:05.844665 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911eb6c-eb4b-4898-981c-f5e61171a7d9 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:05.859155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 641/1232] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:23:05 2022] PUT /identity/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/fa0e659eee5e4a9da8d3d7337cb73798/roles/04bc48cc5b6842029b93a845d6744985 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:23:07.860447 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96fbbf-a841-4398-afa7-896024a8fcba None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:07.861085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96fbbf-a841-4398-afa7-896024a8fcba None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:07.861085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96fbbf-a841-4398-afa7-896024a8fcba None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:07.862596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 592/1233] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:23:07.869117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f02486-671d-4b14-af9d-cfbe92121f5d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:07.869632 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f02486-671d-4b14-af9d-cfbe92121f5d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:07.869632 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f02486-671d-4b14-af9d-cfbe92121f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:07.888717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a3f02486-671d-4b14-af9d-cfbe92121f5d None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:23:07.889903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a3f02486-671d-4b14-af9d-cfbe92121f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:07.960172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 642/1234] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:23:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:23:08.240965 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96fbbf-a841-4398-afa7-896024a8fcba None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:08.270644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:08.360000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96fbbf-a841-4398-afa7-896024a8fcba None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:08.398115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb00486-6355-4f2c-889b-6ccd78fc4128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:08.398746 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb00486-6355-4f2c-889b-6ccd78fc4128 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:08.399021 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb00486-6355-4f2c-889b-6ccd78fc4128 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:08.399286 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb00486-6355-4f2c-889b-6ccd78fc4128 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:08.411328 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb00486-6355-4f2c-889b-6ccd78fc4128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:23:08.416410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-dbb00486-6355-4f2c-889b-6ccd78fc4128 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:23:08.416410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:08.416410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:08.416410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:08.416410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:08.416410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:08.416410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:08.416410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:08.416410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:08.416410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:08.416410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:08.416410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:08.416410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:23:08.416410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:23:08.416410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:23:08.416410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:23:08.417393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:23:08.417393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:23:08.417393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:23:08.417393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:23:08.417393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:08.417393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:08.417393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:08.417393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:08.417393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:08.417393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:08.417393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:08.417393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:08.417393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:08.417393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:08.417393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:08.417393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:08.418222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:08.418222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:08.418222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:08.418222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:08.418222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:23:08.418222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:23:08.418222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:23:08.418222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:23:08.418222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:23:08.418222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:23:08.418222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 29 20:23:08.418222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:23:08.418222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 593/1235] 10.222.0.18 () {56 vars in 1102 bytes} [Mon Aug 29 20:23:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:08.424316 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f02486-671d-4b14-af9d-cfbe92121f5d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:08.474835 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:08.531048 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f02486-671d-4b14-af9d-cfbe92121f5d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:08.566229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b3ef5-a633-4583-a220-0cee14204f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:08.566838 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b3ef5-a633-4583-a220-0cee14204f6c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:08.567115 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b3ef5-a633-4583-a220-0cee14204f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:08.567387 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b3ef5-a633-4583-a220-0cee14204f6c None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:08.568072 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b3ef5-a633-4583-a220-0cee14204f6c None admin] RBAC: Adding query filter params (name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:08.568374 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b3ef5-a633-4583-a220-0cee14204f6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:08.569750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b3ef5-a633-4583-a220-0cee14204f6c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:08.577658 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 643/1236] 10.222.0.18 () {56 vars in 1103 bytes} [Mon Aug 29 20:23:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:08.583510 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb00486-6355-4f2c-889b-6ccd78fc4128 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:08.685098 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:08.704142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb00486-6355-4f2c-889b-6ccd78fc4128 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:08.741711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a815de-8b77-4823-9ad0-26b5e845e563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:08.742413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a815de-8b77-4823-9ad0-26b5e845e563 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:08.742703 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a815de-8b77-4823-9ad0-26b5e845e563 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:08.742963 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a815de-8b77-4823-9ad0-26b5e845e563 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:08.754821 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a815de-8b77-4823-9ad0-26b5e845e563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:08.756305 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a815de-8b77-4823-9ad0-26b5e845e563 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:08.771979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-a3a815de-8b77-4823-9ad0-26b5e845e563 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:08.771979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:08.771979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:08.771979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:08.771979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:08.771979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:08.771979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:08.771979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:08.771979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:08.771979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:08.771979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:08.771979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:08.771979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:23:08.771979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:23:08.771979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:23:08.771979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:23:08.773408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:08.773408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:08.773408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:08.773408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:08.773408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:08.773408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:08.773408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:08.773408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:08.773408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:08.773408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:08.773408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:08.773408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:08.773408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:08.773408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:08.773408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:08.773408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:08.773408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:23:08.774654 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:23:08.774654 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:08.774654 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:23:08.774654 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 594/1237] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:23:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:08.780020 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b3ef5-a633-4583-a220-0cee14204f6c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:08.889600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b3ef5-a633-4583-a220-0cee14204f6c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:08.898707 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:08.928499 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52d3a9-5193-46bf-8d30-67ffb2efee8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:08.929104 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52d3a9-5193-46bf-8d30-67ffb2efee8d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:08.929387 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52d3a9-5193-46bf-8d30-67ffb2efee8d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:08.929660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52d3a9-5193-46bf-8d30-67ffb2efee8d None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:08.930298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e52d3a9-5193-46bf-8d30-67ffb2efee8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:08.930597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e52d3a9-5193-46bf-8d30-67ffb2efee8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:08.932133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e52d3a9-5193-46bf-8d30-67ffb2efee8d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:08.944492 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 644/1238] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:08.951433 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a815de-8b77-4823-9ad0-26b5e845e563 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:09.061803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a815de-8b77-4823-9ad0-26b5e845e563 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:09.097420 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd94a7-1ceb-432d-8e25-bdb55ada9f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:09.098051 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd94a7-1ceb-432d-8e25-bdb55ada9f2f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:09.098328 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd94a7-1ceb-432d-8e25-bdb55ada9f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:09.098597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd94a7-1ceb-432d-8e25-bdb55ada9f2f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:09.102157 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:09.108726 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd94a7-1ceb-432d-8e25-bdb55ada9f2f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:09.110540 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd94a7-1ceb-432d-8e25-bdb55ada9f2f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:09.123943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-53dd94a7-1ceb-432d-8e25-bdb55ada9f2f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:23:09.123943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:09.123943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:09.123943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:09.123943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:09.123943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:09.123943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:09.123943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:09.123943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:09.123943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:09.123943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:09.123943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:09.123943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:23:09.123943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:23:09.123943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:23:09.123943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:23:09.125135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:09.125135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:09.125135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:23:09.125135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:23:09.125135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:23:09.125135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:23:09.125135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:09.125135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:09.125135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:09.125135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:09.125135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:09.125135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:09.125135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:09.125135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:09.125135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:09.125135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:09.125135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:09.125135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:09.126063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:09.126063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:09.126063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:23:09.126063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:23:09.126063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:23:09.126063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:23:09.126063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:23:09.126063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:23:09.126063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:23:09.126063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:23:09.126063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 595/1239] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:23:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:23:09.131599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52d3a9-5193-46bf-8d30-67ffb2efee8d None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:09.259400 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52d3a9-5193-46bf-8d30-67ffb2efee8d None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:09.303284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a27e0-cc3c-4286-bf0e-5ff8444c5873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:09.303994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a27e0-cc3c-4286-bf0e-5ff8444c5873 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:09.304283 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a27e0-cc3c-4286-bf0e-5ff8444c5873 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:09.304714 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a27e0-cc3c-4286-bf0e-5ff8444c5873 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:09.305451 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a27e0-cc3c-4286-bf0e-5ff8444c5873 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:09.305759 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a27e0-cc3c-4286-bf0e-5ff8444c5873 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:09.307179 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a27e0-cc3c-4286-bf0e-5ff8444c5873 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:09.328943 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 645/1240] 10.222.0.18 () {56 vars in 1143 bytes} [Mon Aug 29 20:23:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:23:09.336116 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd94a7-1ceb-432d-8e25-bdb55ada9f2f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:09.458488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd94a7-1ceb-432d-8e25-bdb55ada9f2f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:09.493353 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875081b7-7207-49ff-9841-308d4bdd0857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:09.494007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875081b7-7207-49ff-9841-308d4bdd0857 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:09.494319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875081b7-7207-49ff-9841-308d4bdd0857 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:09.494594 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875081b7-7207-49ff-9841-308d4bdd0857 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:09.570737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:09.581000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875081b7-7207-49ff-9841-308d4bdd0857 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:09.583117 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875081b7-7207-49ff-9841-308d4bdd0857 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:09.701431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 596/1241] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:23:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 29 20:23:09.709274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a27e0-cc3c-4286-bf0e-5ff8444c5873 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:09.749143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:09.838837 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a27e0-cc3c-4286-bf0e-5ff8444c5873 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:09.878544 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ebd11-2cee-4565-b69e-baf3f45f28a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:09.879188 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ebd11-2cee-4565-b69e-baf3f45f28a4 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:09.879471 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ebd11-2cee-4565-b69e-baf3f45f28a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:09.879748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ebd11-2cee-4565-b69e-baf3f45f28a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:09.890985 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ebd11-2cee-4565-b69e-baf3f45f28a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:09.892557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ebd11-2cee-4565-b69e-baf3f45f28a4 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:09.918631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-ac0ebd11-2cee-4565-b69e-baf3f45f28a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:09.918631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:09.918631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:09.918631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:09.918631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:09.918631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:09.918631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:09.918631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:09.918631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:09.918631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:09.918631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:09.918631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:09.918631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:23:09.918631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:23:09.918631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:23:09.918631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:23:09.920926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:09.920926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:09.920926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:09.920926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:09.920926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:09.920926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:09.920926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:09.920926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:09.920926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:09.920926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:09.920926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:09.920926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:09.920926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:09.920926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:09.920926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:09.920926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:09.920926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:23:09.922034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:23:09.922034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:09.922034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:09.923723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 646/1242] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:23:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:23:09.930350 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875081b7-7207-49ff-9841-308d4bdd0857 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:10.034361 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:10.041807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875081b7-7207-49ff-9841-308d4bdd0857 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:10.084664 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb9007-44b4-4f6e-bb0f-930c825350cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:10.085364 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb9007-44b4-4f6e-bb0f-930c825350cc None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:10.085649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb9007-44b4-4f6e-bb0f-930c825350cc None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:10.085923 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb9007-44b4-4f6e-bb0f-930c825350cc None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:10.086585 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb9007-44b4-4f6e-bb0f-930c825350cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:10.086876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb9007-44b4-4f6e-bb0f-930c825350cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:10.088282 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb9007-44b4-4f6e-bb0f-930c825350cc None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:10.104505 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 597/1243] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:10.105566 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ebd11-2cee-4565-b69e-baf3f45f28a4 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:10.190961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:10.220226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ebd11-2cee-4565-b69e-baf3f45f28a4 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:10.258541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c016236-1837-4713-89a8-5d325dca2e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:10.259301 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c016236-1837-4713-89a8-5d325dca2e95 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:10.259613 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c016236-1837-4713-89a8-5d325dca2e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:10.259956 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c016236-1837-4713-89a8-5d325dca2e95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:10.267781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c016236-1837-4713-89a8-5d325dca2e95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:10.269588 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c016236-1837-4713-89a8-5d325dca2e95 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:10.284084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-2c016236-1837-4713-89a8-5d325dca2e95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:23:10.284084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:10.284084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:10.284084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:10.284084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:10.284084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:10.284084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:10.284084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:10.284084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:10.284084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:10.284084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:10.284084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:10.284084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:23:10.284084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:23:10.284084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:23:10.285018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:23:10.285018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:10.285018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:10.285018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:10.285018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:10.285018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:10.285018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:10.285018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:10.285018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:10.285018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:10.285018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:10.285018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:10.285018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:10.285018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:10.285018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:10.285018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:10.285018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:10.286034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:23:10.286034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:23:10.286034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:23:10.286034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:23:10.286034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:23:10.286034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:23:10.286034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:23:10.286034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:10.290227 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 647/1244] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:23:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:23:10.291879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb9007-44b4-4f6e-bb0f-930c825350cc None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:10.415350 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb9007-44b4-4f6e-bb0f-930c825350cc None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:10.461780 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00677e32-24df-4a64-b144-bbf46b9a7e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:10.462540 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00677e32-24df-4a64-b144-bbf46b9a7e95 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:10.462846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00677e32-24df-4a64-b144-bbf46b9a7e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:10.463198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00677e32-24df-4a64-b144-bbf46b9a7e95 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:10.464038 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00677e32-24df-4a64-b144-bbf46b9a7e95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:10.464390 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00677e32-24df-4a64-b144-bbf46b9a7e95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:10.466165 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00677e32-24df-4a64-b144-bbf46b9a7e95 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:10.480952 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 598/1245] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:23:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:23:10.488357 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c016236-1837-4713-89a8-5d325dca2e95 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:10.603728 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c016236-1837-4713-89a8-5d325dca2e95 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:10.655251 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea5753f-a6ba-4712-b25b-2821e5f9cab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:10.655926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5753f-a6ba-4712-b25b-2821e5f9cab1 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:10.656302 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5753f-a6ba-4712-b25b-2821e5f9cab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:10.656673 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5753f-a6ba-4712-b25b-2821e5f9cab1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:10.657582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea5753f-a6ba-4712-b25b-2821e5f9cab1 None admin] RBAC: Adding query filter params (role.id=04bc48cc5b6842029b93a845d6744985, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=fa0e659eee5e4a9da8d3d7337cb73798) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:10.657953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea5753f-a6ba-4712-b25b-2821e5f9cab1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:10.659700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea5753f-a6ba-4712-b25b-2821e5f9cab1 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:10.666452 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 648/1246] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:23:10 2022] GET /identity/v3/role_assignments?user.id=fa0e659eee5e4a9da8d3d7337cb73798&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=04bc48cc5b6842029b93a845d6744985 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:13.031574 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e5ab5-f28e-4f71-b379-6842ec595426 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:13.032316 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e5ab5-f28e-4f71-b379-6842ec595426 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:13.032469 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e5ab5-f28e-4f71-b379-6842ec595426 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:13.033988 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 599/1247] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:23:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:23:13.040144 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333abe9e-f7a4-4d9f-bd19-920964cc105c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:13.040649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333abe9e-f7a4-4d9f-bd19-920964cc105c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:13.040819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333abe9e-f7a4-4d9f-bd19-920964cc105c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:13.047336 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:13.062966 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-333abe9e-f7a4-4d9f-bd19-920964cc105c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:23:13.064688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-333abe9e-f7a4-4d9f-bd19-920964cc105c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:13.163618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 649/1248] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:23:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:23:13.456558 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e5ab5-f28e-4f71-b379-6842ec595426 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:13.478254 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:13.582734 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e5ab5-f28e-4f71-b379-6842ec595426 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:13.632779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4770b44-277c-4b7a-9b80-1054c97eb3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:13.632779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4770b44-277c-4b7a-9b80-1054c97eb3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:13.632779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4770b44-277c-4b7a-9b80-1054c97eb3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:13.633160 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4770b44-277c-4b7a-9b80-1054c97eb3a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:13.648403 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4770b44-277c-4b7a-9b80-1054c97eb3a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:23:13.653612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-d4770b44-277c-4b7a-9b80-1054c97eb3a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:23:13.653612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:13.653612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:13.653612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:13.653612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:13.653612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:13.653612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:13.653612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:13.653612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:13.653612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:13.653612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:13.653612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:13.653612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:23:13.653612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:23:13.653612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:23:13.653612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:23:13.654750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:23:13.654750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:23:13.654750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:23:13.654750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:23:13.654750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:13.654750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:13.654750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:13.654750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:13.654750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:13.654750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:13.654750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:13.654750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:13.654750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:13.654750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:13.654750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:13.654750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:13.656064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:13.656064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:13.656064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:13.656064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:13.656064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:23:13.656064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:23:13.656064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:23:13.656064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:23:13.656064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:23:13.656064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:23:13.656064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:23:13.656064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:23:13.656064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 600/1249] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:23:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:23:13.661939 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333abe9e-f7a4-4d9f-bd19-920964cc105c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:13.797239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333abe9e-f7a4-4d9f-bd19-920964cc105c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:13.843301 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:13.850319 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e6ce0-f7a4-40ff-9394-4967d5ce90b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:13.851338 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e6ce0-f7a4-40ff-9394-4967d5ce90b8 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:13.851825 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e6ce0-f7a4-40ff-9394-4967d5ce90b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:13.852373 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e6ce0-f7a4-40ff-9394-4967d5ce90b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:13.853291 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e6ce0-f7a4-40ff-9394-4967d5ce90b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:13.853698 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e6ce0-f7a4-40ff-9394-4967d5ce90b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:13.855330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e6ce0-f7a4-40ff-9394-4967d5ce90b8 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:13.871903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 650/1250] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:23:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:23:13.873727 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4770b44-277c-4b7a-9b80-1054c97eb3a4 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:13.970023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:14.000910 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4770b44-277c-4b7a-9b80-1054c97eb3a4 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:14.051221 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54efd7-9ea3-4a78-8df4-536cbd2e8e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:14.051996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54efd7-9ea3-4a78-8df4-536cbd2e8e02 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:14.052185 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54efd7-9ea3-4a78-8df4-536cbd2e8e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:14.052464 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54efd7-9ea3-4a78-8df4-536cbd2e8e02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:14.064865 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54efd7-9ea3-4a78-8df4-536cbd2e8e02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:14.066328 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54efd7-9ea3-4a78-8df4-536cbd2e8e02 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:14.082037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-5a54efd7-9ea3-4a78-8df4-536cbd2e8e02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:14.082037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:14.082037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:14.082037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:14.082037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:14.082037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:14.082037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:14.082037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:14.082037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:14.082037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:14.082037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:14.082037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:14.082037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:23:14.082037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:23:14.082037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:23:14.082037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:23:14.083181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:14.083181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:14.083181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:14.083181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:14.083181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:14.083181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:14.083181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:14.083181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:14.083181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:14.083181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:14.083181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:14.083181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:14.083181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:14.083181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:14.083181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:14.083181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:14.083181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:23:14.084252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:23:14.084252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:14.084252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:23:14.084252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 601/1251] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:23:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:14.093247 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e6ce0-f7a4-40ff-9394-4967d5ce90b8 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:14.239738 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e6ce0-f7a4-40ff-9394-4967d5ce90b8 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:14.277171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79363dd6-991a-49a5-aac4-4b326cf46348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:14.277886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79363dd6-991a-49a5-aac4-4b326cf46348 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:14.278170 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79363dd6-991a-49a5-aac4-4b326cf46348 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:14.278470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79363dd6-991a-49a5-aac4-4b326cf46348 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:14.279241 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79363dd6-991a-49a5-aac4-4b326cf46348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:14.279547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79363dd6-991a-49a5-aac4-4b326cf46348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:14.281325 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79363dd6-991a-49a5-aac4-4b326cf46348 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:14.294967 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 651/1252] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:14.302907 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54efd7-9ea3-4a78-8df4-536cbd2e8e02 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:14.426872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54efd7-9ea3-4a78-8df4-536cbd2e8e02 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:14.467184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:14.480231 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf424cf-82be-4d8f-8a69-64f112835343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:14.481161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf424cf-82be-4d8f-8a69-64f112835343 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:14.481472 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf424cf-82be-4d8f-8a69-64f112835343 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:14.481929 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf424cf-82be-4d8f-8a69-64f112835343 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:14.495903 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf424cf-82be-4d8f-8a69-64f112835343 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:14.498155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf424cf-82be-4d8f-8a69-64f112835343 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:14.518928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-bdf424cf-82be-4d8f-8a69-64f112835343 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:23:14.518928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:14.518928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:14.518928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:14.518928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:14.518928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:14.518928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:14.518928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:14.518928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:14.518928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:14.518928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:14.518928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:14.518928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:23:14.518928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:23:14.518928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:23:14.518928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:23:14.520143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:14.520143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:14.520143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:23:14.520143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:23:14.520143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:23:14.520143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:23:14.520143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:14.520143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:14.520143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:14.520143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:14.520143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:14.520143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:14.520143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:14.520143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:14.520143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:14.520143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:14.520143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:14.520143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:14.521228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:14.521228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:14.521228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:23:14.521228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:23:14.521228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:23:14.521228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:23:14.521228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:23:14.521228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:23:14.521228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:23:14.521228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:23:14.521758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 602/1253] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:23:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:14.526992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79363dd6-991a-49a5-aac4-4b326cf46348 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:14.594602 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:14.647543 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79363dd6-991a-49a5-aac4-4b326cf46348 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:14.703605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e719a-c6e6-4bab-b429-b804b8a7c47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:14.704540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e719a-c6e6-4bab-b429-b804b8a7c47a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:14.704936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e719a-c6e6-4bab-b429-b804b8a7c47a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:14.705380 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e719a-c6e6-4bab-b429-b804b8a7c47a None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:14.706671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e719a-c6e6-4bab-b429-b804b8a7c47a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:14.707256 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e719a-c6e6-4bab-b429-b804b8a7c47a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:14.709723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e719a-c6e6-4bab-b429-b804b8a7c47a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:14.735688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 652/1254] 10.222.0.18 () {56 vars in 1143 bytes} [Mon Aug 29 20:23:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:23:14.741771 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf424cf-82be-4d8f-8a69-64f112835343 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:14.868212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf424cf-82be-4d8f-8a69-64f112835343 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:14.909377 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0ef8f-715c-42d7-91f5-be15d2b35faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:14.910189 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0ef8f-715c-42d7-91f5-be15d2b35faa None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:14.910475 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0ef8f-715c-42d7-91f5-be15d2b35faa None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:14.910750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0ef8f-715c-42d7-91f5-be15d2b35faa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:14.984830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:14.999887 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0ef8f-715c-42d7-91f5-be15d2b35faa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:15.001381 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0ef8f-715c-42d7-91f5-be15d2b35faa None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:15.116054 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 603/1255] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:23:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:23:15.123548 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e719a-c6e6-4bab-b429-b804b8a7c47a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:15.243385 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e719a-c6e6-4bab-b429-b804b8a7c47a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:15.261654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:15.287038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbdcd8e-85ef-44ea-b3d0-9e34ca01ff59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:15.287762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdcd8e-85ef-44ea-b3d0-9e34ca01ff59 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:15.288140 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdcd8e-85ef-44ea-b3d0-9e34ca01ff59 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:15.288518 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdcd8e-85ef-44ea-b3d0-9e34ca01ff59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:15.301104 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbdcd8e-85ef-44ea-b3d0-9e34ca01ff59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:15.302948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbdcd8e-85ef-44ea-b3d0-9e34ca01ff59 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:15.321726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-fbbdcd8e-85ef-44ea-b3d0-9e34ca01ff59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:15.321726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:15.321726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:15.321726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:15.321726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:15.321726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:15.321726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:15.321726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:15.321726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:15.321726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:15.321726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:15.321726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:15.321726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:23:15.321726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:23:15.321726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:23:15.321726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:23:15.323036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:15.323036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:15.323036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:15.323036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:15.323036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:15.323036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:15.323036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:15.323036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:15.323036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:15.323036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:15.323036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:15.323036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:15.323036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:15.323036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:15.323036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:15.323036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:15.323036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:23:15.324127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:23:15.324127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:15.324127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:15.324127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 653/1256] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:23:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:15.331463 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0ef8f-715c-42d7-91f5-be15d2b35faa None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:15.430759 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:15.443911 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0ef8f-715c-42d7-91f5-be15d2b35faa None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:15.479666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa783f-87eb-4410-ac15-dd08cea54661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:15.480352 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa783f-87eb-4410-ac15-dd08cea54661 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:15.480629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa783f-87eb-4410-ac15-dd08cea54661 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:15.480899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa783f-87eb-4410-ac15-dd08cea54661 None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:15.481540 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa783f-87eb-4410-ac15-dd08cea54661 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:15.481874 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa783f-87eb-4410-ac15-dd08cea54661 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:15.483987 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa783f-87eb-4410-ac15-dd08cea54661 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:15.498544 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 604/1257] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:15.505291 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbdcd8e-85ef-44ea-b3d0-9e34ca01ff59 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:15.612931 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbdcd8e-85ef-44ea-b3d0-9e34ca01ff59 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:15.652304 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f6028-a521-4163-a0e6-bcf30555574f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:15.653067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f6028-a521-4163-a0e6-bcf30555574f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:15.653356 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f6028-a521-4163-a0e6-bcf30555574f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:15.653638 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f6028-a521-4163-a0e6-bcf30555574f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:15.663965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f6028-a521-4163-a0e6-bcf30555574f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:15.665477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f6028-a521-4163-a0e6-bcf30555574f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:15.678475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-c54f6028-a521-4163-a0e6-bcf30555574f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:23:15.678475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:15.678475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:15.678475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:15.678475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:15.678475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:15.678475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:15.678475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:15.678475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:15.678475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:15.678475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:15.678475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:15.678475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:23:15.678475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:23:15.678475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:23:15.679637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:23:15.679637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:15.679637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:15.679637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:15.679637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:15.679637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:15.679637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:15.679637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:15.679637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:15.679637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:15.679637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:15.679637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:15.679637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:15.679637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:15.679637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:15.679637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:15.679637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:15.681253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:23:15.681253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:23:15.681253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:23:15.681253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:23:15.681253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:23:15.681253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:23:15.681253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:23:15.681253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:15.681253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 654/1258] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:23:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:15.687163 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa783f-87eb-4410-ac15-dd08cea54661 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:15.797599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa783f-87eb-4410-ac15-dd08cea54661 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:15.837831 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2503ff-732f-4502-81de-393d7cee26c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:15.838486 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2503ff-732f-4502-81de-393d7cee26c5 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:15.838768 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2503ff-732f-4502-81de-393d7cee26c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:15.839041 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2503ff-732f-4502-81de-393d7cee26c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:15.839736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2503ff-732f-4502-81de-393d7cee26c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:15.840067 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2503ff-732f-4502-81de-393d7cee26c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:15.842036 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2503ff-732f-4502-81de-393d7cee26c5 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:15.865045 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 605/1259] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:23:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:15.869034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f6028-a521-4163-a0e6-bcf30555574f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:15.897950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:15.996405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f6028-a521-4163-a0e6-bcf30555574f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:16.031904 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cef80-5241-4c14-9177-3b842261ab6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:16.032548 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cef80-5241-4c14-9177-3b842261ab6d None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:16.032830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cef80-5241-4c14-9177-3b842261ab6d None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:16.033107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cef80-5241-4c14-9177-3b842261ab6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:16.033844 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cef80-5241-4c14-9177-3b842261ab6d None admin] RBAC: Adding query filter params (role.id=322da7bd64024770a64bdb73e5c54719, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=fa0e659eee5e4a9da8d3d7337cb73798) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:16.034157 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cef80-5241-4c14-9177-3b842261ab6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:16.035577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cef80-5241-4c14-9177-3b842261ab6d None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:16.041372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 655/1260] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:23:15 2022] GET /identity/v3/role_assignments?user.id=fa0e659eee5e4a9da8d3d7337cb73798&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=322da7bd64024770a64bdb73e5c54719 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:23:17.954824 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca94e84-fd4b-4282-9532-814f2aadc469 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:17.955374 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca94e84-fd4b-4282-9532-814f2aadc469 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:17.955374 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca94e84-fd4b-4282-9532-814f2aadc469 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:17.956761 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 606/1261] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:23:17.961984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50accd24-4abf-4d0f-ab1a-852f6775814c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:17.962381 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50accd24-4abf-4d0f-ab1a-852f6775814c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:17.962538 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50accd24-4abf-4d0f-ab1a-852f6775814c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:17.981112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-50accd24-4abf-4d0f-ab1a-852f6775814c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:23:17.982298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-50accd24-4abf-4d0f-ab1a-852f6775814c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:18.052252 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 656/1262] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:23:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:23:18.317125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca94e84-fd4b-4282-9532-814f2aadc469 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:18.329050 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:18.418094 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca94e84-fd4b-4282-9532-814f2aadc469 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:18.452421 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8352a23-998b-4525-bc4b-ad1dd2f9b1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:18.453046 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8352a23-998b-4525-bc4b-ad1dd2f9b1a1 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:18.453323 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8352a23-998b-4525-bc4b-ad1dd2f9b1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:18.453593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8352a23-998b-4525-bc4b-ad1dd2f9b1a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:18.464231 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8352a23-998b-4525-bc4b-ad1dd2f9b1a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:23:18.469062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-b8352a23-998b-4525-bc4b-ad1dd2f9b1a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:23:18.469062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:18.469062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:18.469062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:18.469062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:18.469062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:18.469062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:18.469062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:18.469062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:18.469062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:18.469062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:18.469062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:18.469062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:23:18.469062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:23:18.469062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:23:18.469062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:23:18.470149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:23:18.470149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:23:18.470149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:23:18.470149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:23:18.470149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:18.470149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:18.470149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:18.470149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:18.470149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:18.470149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:18.470149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:18.470149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:18.470149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:18.470149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:18.470149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:18.470149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:18.471066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:18.471066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:18.471066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:18.471066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:18.471066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:23:18.471066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:23:18.471066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:23:18.471066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:23:18.471066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:23:18.471066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:23:18.471066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:23:18.471066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:23:18.471066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 607/1263] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:23:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:18.476229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50accd24-4abf-4d0f-ab1a-852f6775814c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:18.494433 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:18.589763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50accd24-4abf-4d0f-ab1a-852f6775814c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:18.624905 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb0b7e-ac8b-4140-94fb-5bfcff44ea7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:18.625516 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb0b7e-ac8b-4140-94fb-5bfcff44ea7a None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:18.625796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb0b7e-ac8b-4140-94fb-5bfcff44ea7a None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:18.626067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb0b7e-ac8b-4140-94fb-5bfcff44ea7a None admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:18.626753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb0b7e-ac8b-4140-94fb-5bfcff44ea7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:18.627076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb0b7e-ac8b-4140-94fb-5bfcff44ea7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:18.628488 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb0b7e-ac8b-4140-94fb-5bfcff44ea7a None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:18.636423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 657/1264] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:23:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:18.642337 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8352a23-998b-4525-bc4b-ad1dd2f9b1a1 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:18.715737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:18.751391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8352a23-998b-4525-bc4b-ad1dd2f9b1a1 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:18.790553 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71304cd7-8713-4135-8b1b-6f74f1ff7652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:18.791192 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71304cd7-8713-4135-8b1b-6f74f1ff7652 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:18.791476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71304cd7-8713-4135-8b1b-6f74f1ff7652 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:18.791755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71304cd7-8713-4135-8b1b-6f74f1ff7652 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:18.802981 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71304cd7-8713-4135-8b1b-6f74f1ff7652 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:18.804470 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71304cd7-8713-4135-8b1b-6f74f1ff7652 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:18.828404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-71304cd7-8713-4135-8b1b-6f74f1ff7652 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:18.828404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:18.828404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:18.828404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:18.828404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:18.828404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:18.828404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:18.828404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:18.828404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:18.828404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:18.828404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:18.828404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:18.828404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:23:18.828404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:23:18.828404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:23:18.828404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:23:18.829567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:18.829567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:18.829567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:18.829567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:18.829567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:18.829567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:18.829567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:18.829567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:18.829567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:18.829567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:18.829567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:18.829567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:18.829567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:18.829567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:18.829567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:18.829567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:18.829567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:23:18.830840 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:23:18.830840 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:18.830840 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:23:18.830840 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 608/1265] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:18.839449 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb0b7e-ac8b-4140-94fb-5bfcff44ea7a None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:18.922300 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:18.940768 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb0b7e-ac8b-4140-94fb-5bfcff44ea7a None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:18.973095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef25369-f5f0-4e32-aacc-74868ad62d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:18.973686 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef25369-f5f0-4e32-aacc-74868ad62d37 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:18.973963 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef25369-f5f0-4e32-aacc-74868ad62d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:18.974235 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef25369-f5f0-4e32-aacc-74868ad62d37 None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:18.974874 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef25369-f5f0-4e32-aacc-74868ad62d37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:18.975175 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef25369-f5f0-4e32-aacc-74868ad62d37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:18.976557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef25369-f5f0-4e32-aacc-74868ad62d37 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:18.988597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 658/1266] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:18.994334 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71304cd7-8713-4135-8b1b-6f74f1ff7652 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:19.094789 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71304cd7-8713-4135-8b1b-6f74f1ff7652 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:19.114653 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:19.136796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be0d70-e06b-409d-87a9-935a9b705c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:19.137433 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be0d70-e06b-409d-87a9-935a9b705c55 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:19.137703 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be0d70-e06b-409d-87a9-935a9b705c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:19.137970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be0d70-e06b-409d-87a9-935a9b705c55 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:19.146890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be0d70-e06b-409d-87a9-935a9b705c55 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:19.148357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be0d70-e06b-409d-87a9-935a9b705c55 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:19.160738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-80be0d70-e06b-409d-87a9-935a9b705c55 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:23:19.160738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:19.160738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:19.160738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:19.160738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:19.160738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:19.160738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:19.160738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:19.160738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:19.160738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:19.160738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:19.160738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:19.160738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:23:19.160738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:23:19.160738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:23:19.160738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:23:19.162790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:19.162790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:19.162790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:23:19.162790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:23:19.162790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:23:19.162790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:23:19.162790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:19.162790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:19.162790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:19.162790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:19.162790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:19.162790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:19.162790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:19.162790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:19.162790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:19.162790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:19.162790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:19.162790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:19.164737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:19.164737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:19.164737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:23:19.164737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:23:19.164737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:23:19.164737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:23:19.164737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:23:19.164737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:23:19.164737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:23:19.164737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:23:19.164737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 609/1267] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:23:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:19.167732 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef25369-f5f0-4e32-aacc-74868ad62d37 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:19.266705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef25369-f5f0-4e32-aacc-74868ad62d37 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:19.300580 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37a855-dd36-4a60-a214-1ca640ed1e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:19.301199 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37a855-dd36-4a60-a214-1ca640ed1e23 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:19.301483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37a855-dd36-4a60-a214-1ca640ed1e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:19.301758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37a855-dd36-4a60-a214-1ca640ed1e23 None admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:19.302496 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37a855-dd36-4a60-a214-1ca640ed1e23 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:19.302802 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37a855-dd36-4a60-a214-1ca640ed1e23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:19.304263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37a855-dd36-4a60-a214-1ca640ed1e23 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:19.314000 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:19.323797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 659/1268] 10.222.0.18 () {56 vars in 1143 bytes} [Mon Aug 29 20:23:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:19.329434 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be0d70-e06b-409d-87a9-935a9b705c55 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:19.441693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be0d70-e06b-409d-87a9-935a9b705c55 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:19.473859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d524e-3159-40c4-93f1-645467171ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:19.474478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d524e-3159-40c4-93f1-645467171ace None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:19.474759 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d524e-3159-40c4-93f1-645467171ace None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:19.475033 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d524e-3159-40c4-93f1-645467171ace None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:19.485886 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d524e-3159-40c4-93f1-645467171ace None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:19.487318 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d524e-3159-40c4-93f1-645467171ace None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:19.501744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-524d524e-3159-40c4-93f1-645467171ace None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:19.501744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:19.501744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:19.501744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:19.501744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:19.501744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:19.501744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:19.501744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:19.501744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:19.501744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:19.501744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:19.501744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:19.501744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:23:19.501744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:23:19.501744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:23:19.501744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:23:19.503076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:19.503076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:19.503076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:19.503076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:19.503076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:19.503076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:19.503076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:19.503076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:19.503076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:19.503076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:19.503076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:19.503076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:19.503076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:19.503076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:19.503076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:19.503076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:19.503076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:23:19.504228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:23:19.504228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:19.504228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:23:19.504228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 610/1269] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:23:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:19.508741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37a855-dd36-4a60-a214-1ca640ed1e23 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:19.618942 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37a855-dd36-4a60-a214-1ca640ed1e23 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:19.654111 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4cf073-4aa8-4569-9e0f-1136c940520f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:19.654749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4cf073-4aa8-4569-9e0f-1136c940520f None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:19.655029 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4cf073-4aa8-4569-9e0f-1136c940520f None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:19.655295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4cf073-4aa8-4569-9e0f-1136c940520f None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:19.656232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4cf073-4aa8-4569-9e0f-1136c940520f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:19.656535 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4cf073-4aa8-4569-9e0f-1136c940520f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:19.657892 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4cf073-4aa8-4569-9e0f-1136c940520f None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:19.670732 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 660/1270] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:19.677599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d524e-3159-40c4-93f1-645467171ace None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:19.718287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:19.789363 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d524e-3159-40c4-93f1-645467171ace None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:19.823317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d09a2-8d40-4ced-b48f-a24bfc4217b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:19.823967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d09a2-8d40-4ced-b48f-a24bfc4217b9 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:19.824253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d09a2-8d40-4ced-b48f-a24bfc4217b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:19.824524 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d09a2-8d40-4ced-b48f-a24bfc4217b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:19.831679 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d09a2-8d40-4ced-b48f-a24bfc4217b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:19.833146 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d09a2-8d40-4ced-b48f-a24bfc4217b9 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:19.844360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-b45d09a2-8d40-4ced-b48f-a24bfc4217b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:23:19.844360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:19.844360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:19.844360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:19.844360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:19.844360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:19.844360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:19.844360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:19.844360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:19.844360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:19.844360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:19.844360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:19.844360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:23:19.844360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:23:19.844360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:23:19.845804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:23:19.845804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:19.845804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:19.845804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:19.845804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:19.845804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:19.845804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:19.845804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:19.845804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:19.845804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:19.845804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:19.845804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:19.845804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:19.845804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:19.845804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:19.845804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:19.845804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:19.847148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:23:19.847148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:23:19.847148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:23:19.847148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:23:19.847148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:23:19.847148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:23:19.847148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:23:19.847148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:23:19.847148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 611/1271] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:23:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:19.851389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4cf073-4aa8-4569-9e0f-1136c940520f None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:19.911376 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:19.956107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4cf073-4aa8-4569-9e0f-1136c940520f None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:19.989663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b972fb-4b34-4150-9b15-4415b53910fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:19.990251 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b972fb-4b34-4150-9b15-4415b53910fd None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:19.990529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b972fb-4b34-4150-9b15-4415b53910fd None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:19.990800 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b972fb-4b34-4150-9b15-4415b53910fd None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:19.991486 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b972fb-4b34-4150-9b15-4415b53910fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:19.991790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b972fb-4b34-4150-9b15-4415b53910fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:19.993231 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b972fb-4b34-4150-9b15-4415b53910fd None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:20.004788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 661/1272] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:23:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:20.011129 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d09a2-8d40-4ced-b48f-a24bfc4217b9 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:20.102106 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:20.114069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d09a2-8d40-4ced-b48f-a24bfc4217b9 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:20.147123 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c99906-2fd6-4c5d-853a-6dd9d51e1b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:20.147771 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c99906-2fd6-4c5d-853a-6dd9d51e1b03 None admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:20.148199 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c99906-2fd6-4c5d-853a-6dd9d51e1b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:20.148482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c99906-2fd6-4c5d-853a-6dd9d51e1b03 None admin] PATH_INFO: `/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/fa0e659eee5e4a9da8d3d7337cb73798/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:20.160727 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c99906-2fd6-4c5d-853a-6dd9d51e1b03 None admin] RBAC: Authorizing `identity:create_grant(project_id=90b25dc95d5a4a068994bdd801d6b8ea, user_id=fa0e659eee5e4a9da8d3d7337cb73798, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:20.162256 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c99906-2fd6-4c5d-853a-6dd9d51e1b03 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:20.175544 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 612/1273] 10.222.0.18 () {58 vars in 1439 bytes} [Mon Aug 29 20:23:20 2022] PUT /identity/v3/projects/90b25dc95d5a4a068994bdd801d6b8ea/users/fa0e659eee5e4a9da8d3d7337cb73798/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:23:22.115602 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d2b76-ba6e-4806-a3b9-15a53c1fc947 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:22.116187 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d2b76-ba6e-4806-a3b9-15a53c1fc947 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:22.116187 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d2b76-ba6e-4806-a3b9-15a53c1fc947 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:22.117642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 662/1274] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:23:22.122990 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eae5b4-542c-4120-a549-41bee1d6c0bf None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:22.123534 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eae5b4-542c-4120-a549-41bee1d6c0bf None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:22.123660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eae5b4-542c-4120-a549-41bee1d6c0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:22.146166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-f1eae5b4-542c-4120-a549-41bee1d6c0bf None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:23:22.147365 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f1eae5b4-542c-4120-a549-41bee1d6c0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:22.222185 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 613/1275] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:23:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:23:22.489528 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d2b76-ba6e-4806-a3b9-15a53c1fc947 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:22.600708 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d2b76-ba6e-4806-a3b9-15a53c1fc947 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:22.618236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:22.635573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553c0e9-c1f2-490b-8b90-fcd5ce3f9cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:22.636240 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553c0e9-c1f2-490b-8b90-fcd5ce3f9cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:22.636521 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553c0e9-c1f2-490b-8b90-fcd5ce3f9cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:22.636792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553c0e9-c1f2-490b-8b90-fcd5ce3f9cc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:22.652743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553c0e9-c1f2-490b-8b90-fcd5ce3f9cc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 29 20:23:22.659990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-0553c0e9-c1f2-490b-8b90-fcd5ce3f9cc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:23:22.659990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:22.659990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:22.659990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:22.659990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:22.659990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:22.659990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:22.659990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:22.659990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:22.659990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:22.659990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:22.659990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:22.659990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 29 20:23:22.659990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 29 20:23:22.659990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 29 20:23:22.659990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 29 20:23:22.661206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 29 20:23:22.661206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 29 20:23:22.661206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 29 20:23:22.661206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 29 20:23:22.661206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:22.661206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:22.661206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:22.661206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:22.661206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:22.661206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:22.661206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:22.661206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:22.661206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:22.661206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:22.661206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:22.661206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:22.662216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:22.662216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:22.662216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:22.662216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:22.662216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 29 20:23:22.662216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 29 20:23:22.662216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 29 20:23:22.662216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 29 20:23:22.662216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 29 20:23:22.662216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 29 20:23:22.662216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 29 20:23:22.662216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:22.662216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 663/1276] 10.222.0.18 () {56 vars in 1096 bytes} [Mon Aug 29 20:23:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:22.667266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eae5b4-542c-4120-a549-41bee1d6c0bf None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:22.777213 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:22.796987 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eae5b4-542c-4120-a549-41bee1d6c0bf None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:22.832438 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ce9a8-e62a-4169-989a-fc860b794f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:22.833049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ce9a8-e62a-4169-989a-fc860b794f6f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:22.833330 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ce9a8-e62a-4169-989a-fc860b794f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:22.833604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ce9a8-e62a-4169-989a-fc860b794f6f None admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:22.834305 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ce9a8-e62a-4169-989a-fc860b794f6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:22.834611 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ce9a8-e62a-4169-989a-fc860b794f6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:22.836012 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ce9a8-e62a-4169-989a-fc860b794f6f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:22.844350 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 614/1277] 10.222.0.18 () {56 vars in 1099 bytes} [Mon Aug 29 20:23:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:22.850097 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553c0e9-c1f2-490b-8b90-fcd5ce3f9cc2 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:22.952680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553c0e9-c1f2-490b-8b90-fcd5ce3f9cc2 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:22.986568 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20590dd-e19b-4844-a9af-9737e22761d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:22.987201 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20590dd-e19b-4844-a9af-9737e22761d3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:22.987481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20590dd-e19b-4844-a9af-9737e22761d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:22.987751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20590dd-e19b-4844-a9af-9737e22761d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:22.997786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20590dd-e19b-4844-a9af-9737e22761d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:22.999926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20590dd-e19b-4844-a9af-9737e22761d3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:23.014915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-b20590dd-e19b-4844-a9af-9737e22761d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:23.014915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:23.014915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:23.014915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:23.014915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:23.014915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:23.014915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:23.014915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:23.014915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:23.014915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:23.014915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:23.014915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:23.014915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:23:23.014915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:23:23.014915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:23:23.014915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:23:23.016213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:23.016213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:23.016213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:23.016213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:23.016213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:23.016213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:23.016213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:23.016213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:23.016213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:23.016213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:23.016213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:23.016213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:23.016213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:23.016213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:23.016213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:23.016213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:23.016213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:23:23.017210 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:23:23.017210 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:23.017210 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:23.017210 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 664/1278] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:23:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:23.022105 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ce9a8-e62a-4169-989a-fc860b794f6f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:23.121198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ce9a8-e62a-4169-989a-fc860b794f6f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:23.154284 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b4217-d2a2-4380-abb0-61fc5682dd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:23.155534 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b4217-d2a2-4380-abb0-61fc5682dd0c None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:23.155721 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b4217-d2a2-4380-abb0-61fc5682dd0c None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:23.156016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b4217-d2a2-4380-abb0-61fc5682dd0c None admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:23.156917 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b4217-d2a2-4380-abb0-61fc5682dd0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:23.157216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b4217-d2a2-4380-abb0-61fc5682dd0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:23.158579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b4217-d2a2-4380-abb0-61fc5682dd0c None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:23.170178 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 615/1279] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:23.175971 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20590dd-e19b-4844-a9af-9737e22761d3 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:23.206403 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:23.274060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20590dd-e19b-4844-a9af-9737e22761d3 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:23.312563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5f028-5ec5-4139-a9bb-0728fbeff797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:23.313201 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5f028-5ec5-4139-a9bb-0728fbeff797 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:23.313513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5f028-5ec5-4139-a9bb-0728fbeff797 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:23.313787 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5f028-5ec5-4139-a9bb-0728fbeff797 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:23.321866 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5f028-5ec5-4139-a9bb-0728fbeff797 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:23.323355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5f028-5ec5-4139-a9bb-0728fbeff797 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:23.336436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-dca5f028-5ec5-4139-a9bb-0728fbeff797 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:23:23.336436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:23.336436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:23.336436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:23.336436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:23.336436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:23.336436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:23.336436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:23.336436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:23.336436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:23.336436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:23.336436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:23.336436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 29 20:23:23.336436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 29 20:23:23.336436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 29 20:23:23.336436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 29 20:23:23.338646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:23.338646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:23.338646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 29 20:23:23.338646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:23:23.338646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 29 20:23:23.338646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 29 20:23:23.338646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:23.338646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:23.338646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:23.338646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:23.338646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:23.338646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:23.338646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:23.338646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:23.338646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:23.338646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:23.338646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:23.338646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:23.340133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:23.340133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:23.340133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 29 20:23:23.340133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 29 20:23:23.340133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 29 20:23:23.340133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 29 20:23:23.340133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 29 20:23:23.340133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 29 20:23:23.340133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 29 20:23:23.340133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:23.340133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 665/1280] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:23:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:23.344107 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b4217-d2a2-4380-abb0-61fc5682dd0c None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:23.348038 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:23.445071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b4217-d2a2-4380-abb0-61fc5682dd0c None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:23.479293 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9af10-584c-493e-9c09-110040d3f3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:23.479895 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9af10-584c-493e-9c09-110040d3f3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:23.480236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9af10-584c-493e-9c09-110040d3f3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:23.480510 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9af10-584c-493e-9c09-110040d3f3d4 None admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:23.481235 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9af10-584c-493e-9c09-110040d3f3d4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:23.481537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9af10-584c-493e-9c09-110040d3f3d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:23.482959 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9af10-584c-493e-9c09-110040d3f3d4 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:23.500435 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 616/1281] 10.222.0.18 () {56 vars in 1143 bytes} [Mon Aug 29 20:23:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:23.506301 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5f028-5ec5-4139-a9bb-0728fbeff797 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:23.596533 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:23.606370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5f028-5ec5-4139-a9bb-0728fbeff797 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:23.643474 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17a5c3-c747-4a8a-b9fc-1ff8a891955c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:23.644122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17a5c3-c747-4a8a-b9fc-1ff8a891955c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:23.644407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17a5c3-c747-4a8a-b9fc-1ff8a891955c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:23.644684 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17a5c3-c747-4a8a-b9fc-1ff8a891955c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:23.718701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17a5c3-c747-4a8a-b9fc-1ff8a891955c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:23.720509 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17a5c3-c747-4a8a-b9fc-1ff8a891955c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:23.810441 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:23.822184 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 666/1282] 10.222.0.18 () {58 vars in 1282 bytes} [Mon Aug 29 20:23:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 29 20:23:23.828086 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9af10-584c-493e-9c09-110040d3f3d4 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:23.888028 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:23.926701 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9af10-584c-493e-9c09-110040d3f3d4 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:23.960436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4d553-92dd-49dd-be7d-d6dc323800b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:23.961066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4d553-92dd-49dd-be7d-d6dc323800b1 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:23.961346 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4d553-92dd-49dd-be7d-d6dc323800b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:23.961625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4d553-92dd-49dd-be7d-d6dc323800b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:23.972787 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4d553-92dd-49dd-be7d-d6dc323800b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:23.974364 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4d553-92dd-49dd-be7d-d6dc323800b1 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:23.991071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-35b4d553-92dd-49dd-be7d-d6dc323800b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:23.991071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:23.991071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:23.991071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:23.991071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:23.991071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:23.991071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:23.991071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:23.991071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:23.991071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:23.991071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:23.991071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:23.991071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 29 20:23:23.991071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 29 20:23:23.991071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 29 20:23:23.991071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 29 20:23:23.992115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:23.992115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:23.992115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:23.992115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:23.992115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:23.992115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:23.992115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:23.992115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:23.992115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:23.992115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:23.992115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:23.992115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:23.992115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:23.992115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:23.992115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:23.992115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:23.992115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 29 20:23:23.994139 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 29 20:23:23.994139 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 29 20:23:23.994139 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:23:23.994139 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 617/1283] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:23:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:23.999026 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17a5c3-c747-4a8a-b9fc-1ff8a891955c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:24.101361 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17a5c3-c747-4a8a-b9fc-1ff8a891955c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:24.135112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa1f7c-ea44-487f-9616-18743bbce8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:24.135727 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa1f7c-ea44-487f-9616-18743bbce8ca None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:24.136059 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa1f7c-ea44-487f-9616-18743bbce8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:24.136366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa1f7c-ea44-487f-9616-18743bbce8ca None admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:24.137058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa1f7c-ea44-487f-9616-18743bbce8ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:24.137364 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa1f7c-ea44-487f-9616-18743bbce8ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:24.138732 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa1f7c-ea44-487f-9616-18743bbce8ca None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:24.152480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 667/1284] 10.222.0.18 () {56 vars in 1109 bytes} [Mon Aug 29 20:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:24.159257 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4d553-92dd-49dd-be7d-d6dc323800b1 None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:24.261188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4d553-92dd-49dd-be7d-d6dc323800b1 None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:24.275072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:24.295649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e7413-54d8-462e-afd5-3f873babfdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:24.296376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e7413-54d8-462e-afd5-3f873babfdaa None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:24.296597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e7413-54d8-462e-afd5-3f873babfdaa None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:24.296865 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e7413-54d8-462e-afd5-3f873babfdaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:24.303628 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e7413-54d8-462e-afd5-3f873babfdaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:24.305165 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805e7413-54d8-462e-afd5-3f873babfdaa None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:24.316950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application [None req-805e7413-54d8-462e-afd5-3f873babfdaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:23:24.316950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:24.316950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:24.316950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:24.316950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:24.316950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:24.316950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:24.316950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:24.316950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:24.316950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:24.316950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:24.316950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:24.316950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 29 20:23:24.316950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 29 20:23:24.316950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 29 20:23:24.318060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 29 20:23:24.318060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:24.318060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:24.318060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:24.318060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:24.318060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:24.318060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:24.318060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:24.318060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:24.318060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:24.318060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:24.318060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:24.318060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:24.318060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:24.318060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:24.318060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:24.318060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:24.319158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 29 20:23:24.319158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 29 20:23:24.319158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 29 20:23:24.319158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 29 20:23:24.319158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 29 20:23:24.319158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 29 20:23:24.319158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 29 20:23:24.319158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: ERROR keystone.server.flask.application Aug 29 20:23:24.319158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 618/1285] 10.222.0.18 () {56 vars in 1111 bytes} [Mon Aug 29 20:23:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:24.324034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa1f7c-ea44-487f-9616-18743bbce8ca None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:24.373182 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:24.425127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa1f7c-ea44-487f-9616-18743bbce8ca None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:24.459762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87566d76-c29a-4cd9-9441-b0ed0d61cc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:24.460457 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87566d76-c29a-4cd9-9441-b0ed0d61cc96 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:24.460776 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87566d76-c29a-4cd9-9441-b0ed0d61cc96 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:24.461056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87566d76-c29a-4cd9-9441-b0ed0d61cc96 None admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:24.461753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87566d76-c29a-4cd9-9441-b0ed0d61cc96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:24.462081 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87566d76-c29a-4cd9-9441-b0ed0d61cc96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:24.463503 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87566d76-c29a-4cd9-9441-b0ed0d61cc96 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:24.476120 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 668/1286] 10.222.0.18 () {56 vars in 1148 bytes} [Mon Aug 29 20:23:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:24.483441 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e7413-54d8-462e-afd5-3f873babfdaa None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:24.588663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805e7413-54d8-462e-afd5-3f873babfdaa None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:24.621947 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd8d38-f62c-46d7-b975-e6133d2f3fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:24.622556 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd8d38-f62c-46d7-b975-e6133d2f3fbd None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:24.622828 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd8d38-f62c-46d7-b975-e6133d2f3fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:24.623098 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd8d38-f62c-46d7-b975-e6133d2f3fbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:24.624403 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd8d38-f62c-46d7-b975-e6133d2f3fbd None admin] RBAC: Adding query filter params (role.id=322da7bd64024770a64bdb73e5c54719, scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea, user.id=fa0e659eee5e4a9da8d3d7337cb73798) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:24.624712 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd8d38-f62c-46d7-b975-e6133d2f3fbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:24.626146 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd8d38-f62c-46d7-b975-e6133d2f3fbd None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:24.632325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 619/1287] 10.222.0.18 () {56 vars in 1374 bytes} [Mon Aug 29 20:23:24 2022] GET /identity/v3/role_assignments?user.id=fa0e659eee5e4a9da8d3d7337cb73798&scope.project.id=90b25dc95d5a4a068994bdd801d6b8ea&role.id=322da7bd64024770a64bdb73e5c54719 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:26.570639 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887aa35-bca3-4e60-af88-5b9232e0779d None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:26.571145 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887aa35-bca3-4e60-af88-5b9232e0779d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:26.571145 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887aa35-bca3-4e60-af88-5b9232e0779d None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:26.572485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 669/1288] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:23:26.577773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0aaa3-6f7a-459d-8fb9-abbdb05fea63 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:26.578166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0aaa3-6f7a-459d-8fb9-abbdb05fea63 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:26.578329 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0aaa3-6f7a-459d-8fb9-abbdb05fea63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:26.596731 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-efc0aaa3-6f7a-459d-8fb9-abbdb05fea63 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:23:26.597928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-efc0aaa3-6f7a-459d-8fb9-abbdb05fea63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:26.627163 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:26.664849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 620/1289] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:23:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:23:26.947727 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887aa35-bca3-4e60-af88-5b9232e0779d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:27.059211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887aa35-bca3-4e60-af88-5b9232e0779d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:27.071482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:27.095743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85124d33-86c2-4136-8526-7391e8db405c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:27.096693 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85124d33-86c2-4136-8526-7391e8db405c None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:27.096977 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85124d33-86c2-4136-8526-7391e8db405c None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:27.097252 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85124d33-86c2-4136-8526-7391e8db405c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:27.097833 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85124d33-86c2-4136-8526-7391e8db405c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:27.099219 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85124d33-86c2-4136-8526-7391e8db405c None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:27.110330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-85124d33-86c2-4136-8526-7391e8db405c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:23:27.110330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:27.110330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:23:27.110330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:23:27.110330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:23:27.110330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:23:27.110330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:23:27.110330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:23:27.110330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:23:27.110330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:27.110330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:23:27.110330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:27.110330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:27.110330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:27.110330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:27.110330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:27.110330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:27.111627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:27.111627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:27.111627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:27.111627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:27.111627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:27.111627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:27.111627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:23:27.111627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:23:27.111627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:23:27.111627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:23:27.111627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:27.111627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:27.111627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:27.111627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:27.111627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:27.111627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:27.112805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:27.112805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:27.112805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:27.112805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:27.112805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:27.112805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:27.112805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:27.112805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:27.112805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:27.112805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:27.112805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:23:27.112805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:23:27.112805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:23:27.112805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:27.112805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 670/1290] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Aug 29 20:23:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:27.118667 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc0aaa3-6f7a-459d-8fb9-abbdb05fea63 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:27.214739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc0aaa3-6f7a-459d-8fb9-abbdb05fea63 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:27.249251 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc71011-9e9f-4a40-a1fa-87419f304d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:27.249850 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc71011-9e9f-4a40-a1fa-87419f304d1a None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:27.250129 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc71011-9e9f-4a40-a1fa-87419f304d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:27.250402 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc71011-9e9f-4a40-a1fa-87419f304d1a None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:27.251026 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc71011-9e9f-4a40-a1fa-87419f304d1a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:27.251327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc71011-9e9f-4a40-a1fa-87419f304d1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:27.252722 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc71011-9e9f-4a40-a1fa-87419f304d1a None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:27.257488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 621/1291] 10.222.0.18 () {56 vars in 1116 bytes} [Mon Aug 29 20:23:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:27.263289 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85124d33-86c2-4136-8526-7391e8db405c None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:27.365955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85124d33-86c2-4136-8526-7391e8db405c None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:27.418980 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037a1ae-24e7-4402-8a49-53cba0ff5fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:27.419688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037a1ae-24e7-4402-8a49-53cba0ff5fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:27.420005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037a1ae-24e7-4402-8a49-53cba0ff5fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:27.420287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037a1ae-24e7-4402-8a49-53cba0ff5fa6 None admin] PATH_INFO: `/v3/services` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:27.421537 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037a1ae-24e7-4402-8a49-53cba0ff5fa6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:27.421851 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037a1ae-24e7-4402-8a49-53cba0ff5fa6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:27.423237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037a1ae-24e7-4402-8a49-53cba0ff5fa6 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:27.430086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 671/1292] 10.222.0.18 () {56 vars in 1116 bytes} [Mon Aug 29 20:23:27 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:29.375701 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e2554-a94d-4e72-ac34-91f2680fcfc0 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:29.376635 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e2554-a94d-4e72-ac34-91f2680fcfc0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:29.376635 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e2554-a94d-4e72-ac34-91f2680fcfc0 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:29.377736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 622/1293] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:23:29.382972 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628871c9-6158-44af-a3ca-c5ad96e0adbb None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:29.383426 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628871c9-6158-44af-a3ca-c5ad96e0adbb None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:29.383526 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628871c9-6158-44af-a3ca-c5ad96e0adbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:29.401692 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-628871c9-6158-44af-a3ca-c5ad96e0adbb None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:23:29.402909 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-628871c9-6158-44af-a3ca-c5ad96e0adbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:29.433837 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:29.476120 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 672/1294] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:23:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:23:29.775370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e2554-a94d-4e72-ac34-91f2680fcfc0 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:29.807172 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:29.888949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e2554-a94d-4e72-ac34-91f2680fcfc0 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:29.924551 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46f839-1992-49d4-9313-1220a66c7e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:29.925165 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46f839-1992-49d4-9313-1220a66c7e53 None admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:29.925442 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46f839-1992-49d4-9313-1220a66c7e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:29.925712 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46f839-1992-49d4-9313-1220a66c7e53 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:29.926366 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46f839-1992-49d4-9313-1220a66c7e53 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:29.927752 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46f839-1992-49d4-9313-1220a66c7e53 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:29.943191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 623/1295] 10.222.0.18 () {60 vars in 1141 bytes} [Mon Aug 29 20:23:29 2022] POST /identity/v3/services => generated 246 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:23:31.902654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1efc6-13ee-4003-a497-2a10c07b2be0 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:31.903205 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1efc6-13ee-4003-a497-2a10c07b2be0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:31.903205 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1efc6-13ee-4003-a497-2a10c07b2be0 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:31.904683 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 673/1296] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:23:31.910809 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051f8d8-7726-4df2-aad9-8ab769e84e7d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:31.911244 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051f8d8-7726-4df2-aad9-8ab769e84e7d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:31.911370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051f8d8-7726-4df2-aad9-8ab769e84e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:31.930779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6051f8d8-7726-4df2-aad9-8ab769e84e7d None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:23:31.932005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6051f8d8-7726-4df2-aad9-8ab769e84e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:32.004198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 624/1297] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:23:31 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 29 20:23:32.275294 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1efc6-13ee-4003-a497-2a10c07b2be0 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:32.382524 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1efc6-13ee-4003-a497-2a10c07b2be0 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:32.417728 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b7531-4bd5-4d78-84f4-d41d6bf7bf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:32.418385 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b7531-4bd5-4d78-84f4-d41d6bf7bf43 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:32.418710 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b7531-4bd5-4d78-84f4-d41d6bf7bf43 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:32.418988 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b7531-4bd5-4d78-84f4-d41d6bf7bf43 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:32.419628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b7531-4bd5-4d78-84f4-d41d6bf7bf43 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:32.421091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b7531-4bd5-4d78-84f4-d41d6bf7bf43 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:32.430391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-348b7531-4bd5-4d78-84f4-d41d6bf7bf43 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:23:32.430391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:32.430391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:23:32.430391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:23:32.430391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:23:32.430391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:23:32.430391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:23:32.430391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:23:32.430391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:23:32.430391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:32.430391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:23:32.430391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:32.430391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:32.430391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:32.430391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:32.430391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:32.430391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:32.431618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:32.431618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:32.431618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:32.431618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:32.431618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:32.431618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:32.431618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:23:32.431618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:23:32.431618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:23:32.431618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:23:32.431618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:32.431618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:32.431618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:32.431618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:32.431618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:32.431618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:32.432554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:32.432554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:32.432554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:32.432554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:32.432554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:32.432554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:32.432554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:32.432554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:32.432554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:32.432554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:32.432554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:23:32.432554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:23:32.432554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:23:32.432554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:32.432554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 674/1298] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Aug 29 20:23:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 29 20:23:32.438243 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051f8d8-7726-4df2-aad9-8ab769e84e7d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:32.456611 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:32.538023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051f8d8-7726-4df2-aad9-8ab769e84e7d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:32.571108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377efb6-c337-4d04-9bfe-fccc6fe3469f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:32.571750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377efb6-c337-4d04-9bfe-fccc6fe3469f None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:32.572082 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377efb6-c337-4d04-9bfe-fccc6fe3469f None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:32.572368 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377efb6-c337-4d04-9bfe-fccc6fe3469f None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:32.573008 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377efb6-c337-4d04-9bfe-fccc6fe3469f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:32.573334 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377efb6-c337-4d04-9bfe-fccc6fe3469f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:32.574835 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377efb6-c337-4d04-9bfe-fccc6fe3469f None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:32.580933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 625/1299] 10.222.0.18 () {56 vars in 1116 bytes} [Mon Aug 29 20:23:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:32.587209 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b7531-4bd5-4d78-84f4-d41d6bf7bf43 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:32.619559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:32.698866 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b7531-4bd5-4d78-84f4-d41d6bf7bf43 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:32.740668 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f220af6-eabb-409c-a585-cd26a1d5f673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:32.741112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f220af6-eabb-409c-a585-cd26a1d5f673 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:32.741391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f220af6-eabb-409c-a585-cd26a1d5f673 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:32.741656 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f220af6-eabb-409c-a585-cd26a1d5f673 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:32.742370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f220af6-eabb-409c-a585-cd26a1d5f673 None admin] RBAC: Adding query filter params (interface=public, service_id=d0424f609b1f4910a1761e28d46fac44, region_id=RegionOne) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:32.742670 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f220af6-eabb-409c-a585-cd26a1d5f673 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:32.744051 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f220af6-eabb-409c-a585-cd26a1d5f673 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:32.750126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 675/1300] 10.222.0.18 () {56 vars in 1251 bytes} [Mon Aug 29 20:23:32 2022] GET /identity/v3/endpoints?service_id=d0424f609b1f4910a1761e28d46fac44&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:23:32.757969 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377efb6-c337-4d04-9bfe-fccc6fe3469f None admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:32.851790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:32.871283 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377efb6-c337-4d04-9bfe-fccc6fe3469f None admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:32.906098 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705598db-ae25-420a-8476-e92a52102054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:32.906738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705598db-ae25-420a-8476-e92a52102054 None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:32.907028 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705598db-ae25-420a-8476-e92a52102054 None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:32.907301 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705598db-ae25-420a-8476-e92a52102054 None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:32.908231 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705598db-ae25-420a-8476-e92a52102054 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:32.909619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705598db-ae25-420a-8476-e92a52102054 None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:32.916709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 626/1301] 10.222.0.18 () {56 vars in 1087 bytes} [Mon Aug 29 20:23:32 2022] GET /identity/v3/services => generated 2191 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:23:34.848316 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffd8c2-3384-40ad-9ae4-fa4260d6e9ff None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:34.848909 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffd8c2-3384-40ad-9ae4-fa4260d6e9ff None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:34.848909 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffd8c2-3384-40ad-9ae4-fa4260d6e9ff None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:34.850297 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 676/1302] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:23:34.855604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daf86c-6a0a-4e7d-85ed-5142e1e7f08a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:34.856061 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daf86c-6a0a-4e7d-85ed-5142e1e7f08a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:34.856182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7daf86c-6a0a-4e7d-85ed-5142e1e7f08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:34.875123 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e7daf86c-6a0a-4e7d-85ed-5142e1e7f08a None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:23:34.876379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e7daf86c-6a0a-4e7d-85ed-5142e1e7f08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:23:34.947443 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 627/1303] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:23:34 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 29 20:23:35.214857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ffd8c2-3384-40ad-9ae4-fa4260d6e9ff None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:35.331861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:35.339174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ffd8c2-3384-40ad-9ae4-fa4260d6e9ff None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:35.374443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891ce6a-9218-4971-ba35-09623619c0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:35.375663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891ce6a-9218-4971-ba35-09623619c0b3 None admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:35.375984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891ce6a-9218-4971-ba35-09623619c0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:35.376292 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891ce6a-9218-4971-ba35-09623619c0b3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:35.376899 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891ce6a-9218-4971-ba35-09623619c0b3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:35.378373 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891ce6a-9218-4971-ba35-09623619c0b3 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:35.387749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application [None req-0891ce6a-9218-4971-ba35-09623619c0b3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:23:35.387749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:35.387749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 29 20:23:35.387749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 29 20:23:35.387749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 29 20:23:35.387749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 29 20:23:35.387749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 29 20:23:35.387749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:23:35.387749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:23:35.387749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:35.387749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 29 20:23:35.387749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:35.387749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 29 20:23:35.387749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 29 20:23:35.387749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 29 20:23:35.387749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 29 20:23:35.387749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 29 20:23:35.389088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 29 20:23:35.389088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 29 20:23:35.389088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 29 20:23:35.389088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 29 20:23:35.389088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 29 20:23:35.389088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 29 20:23:35.389088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 29 20:23:35.389088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 29 20:23:35.389088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 29 20:23:35.389088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 29 20:23:35.389088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 29 20:23:35.389088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 29 20:23:35.389088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 29 20:23:35.389088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 29 20:23:35.389088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 29 20:23:35.389088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.get_or_create( Aug 29 20:23:35.390875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 29 20:23:35.390875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application with Lock( Aug 29 20:23:35.390875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 29 20:23:35.390875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self._enter() Aug 29 20:23:35.390875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 29 20:23:35.390875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 29 20:23:35.390875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 29 20:23:35.390875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application return self.creator() Aug 29 20:23:35.390875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 29 20:23:35.390875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application created_value = creator( Aug 29 20:23:35.390875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 29 20:23:35.390875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 29 20:23:35.390875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 29 20:23:35.390875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: ERROR keystone.server.flask.application Aug 29 20:23:35.390875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 677/1304] 10.222.0.18 () {56 vars in 1117 bytes} [Mon Aug 29 20:23:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 29 20:23:35.395841 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7daf86c-6a0a-4e7d-85ed-5142e1e7f08a None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:35.471241 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:35.495578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7daf86c-6a0a-4e7d-85ed-5142e1e7f08a None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:35.531486 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ecc01-acad-4f33-bd00-14a2631404dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:35.532141 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ecc01-acad-4f33-bd00-14a2631404dc None admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:35.532495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ecc01-acad-4f33-bd00-14a2631404dc None admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:35.533042 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ecc01-acad-4f33-bd00-14a2631404dc None admin] PATH_INFO: `/v3/services` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:35.533758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ecc01-acad-4f33-bd00-14a2631404dc None admin] RBAC: Adding query filter params (name=placement) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:23:35.534075 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ecc01-acad-4f33-bd00-14a2631404dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:35.535566 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ecc01-acad-4f33-bd00-14a2631404dc None admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:35.541270 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 628/1305] 10.222.0.18 () {56 vars in 1116 bytes} [Mon Aug 29 20:23:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:23:35.550248 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891ce6a-9218-4971-ba35-09623619c0b3 None admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:23:35.652158 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891ce6a-9218-4971-ba35-09623619c0b3 None admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:23:35.688242 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb470c-a733-4497-94d1-db92272bec50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:23:35.688861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb470c-a733-4497-94d1-db92272bec50 None admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:23:35.689213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb470c-a733-4497-94d1-db92272bec50 None admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:23:35.689456 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb470c-a733-4497-94d1-db92272bec50 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:23:35.690064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb470c-a733-4497-94d1-db92272bec50 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:23:35.691411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb470c-a733-4497-94d1-db92272bec50 None admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:23:35.710814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 678/1306] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:23:35 2022] POST /identity/v3/endpoints => generated 331 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:23:45.537779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=30 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:23:45.706828 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:24:40.236716 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d832c1-fea9-492e-88e7-dcf2d7262fe3 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:40.237296 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d832c1-fea9-492e-88e7-dcf2d7262fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:40.237296 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d832c1-fea9-492e-88e7-dcf2d7262fe3 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:40.242750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 629/1307] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:24:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:24:40.244448 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebb53b-d2f3-4215-a120-201126cb4d56 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:40.245320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebb53b-d2f3-4215-a120-201126cb4d56 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:40.246034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebb53b-d2f3-4215-a120-201126cb4d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:40.394913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-7bebb53b-d2f3-4215-a120-201126cb4d56 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:24:40.396619 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7bebb53b-d2f3-4215-a120-201126cb4d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:40.677061 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 679/1308] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:24:41.001015 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e84c4-0067-4696-8d82-0f807843f15b None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:41.001577 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e84c4-0067-4696-8d82-0f807843f15b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:41.002567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e84c4-0067-4696-8d82-0f807843f15b None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:41.004116 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 630/1309] 10.222.0.18 () {54 vars in 965 bytes} [Mon Aug 29 20:24:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:24:41.010640 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67258c18-2e69-457c-a4f9-c7a0c42603a2 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:41.011008 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67258c18-2e69-457c-a4f9-c7a0c42603a2 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:41.011363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67258c18-2e69-457c-a4f9-c7a0c42603a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:41.094993 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-67258c18-2e69-457c-a4f9-c7a0c42603a2 None None] MFA Rules not processed for user `d51e331a4c9c4f038bdb80f8c5b74e42`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:24:41.096349 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-67258c18-2e69-457c-a4f9-c7a0c42603a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:41.119500 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:24:41.195986 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 680/1310] 10.222.0.18 () {58 vars in 1062 bytes} [Mon Aug 29 20:24:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:24:41.206188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e84c4-0067-4696-8d82-0f807843f15b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:24:41.207972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a37e84c4-0067-4696-8d82-0f807843f15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:41.372476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e84c4-0067-4696-8d82-0f807843f15b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:24:41.466459 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc636e-4848-4e18-aecc-31e65ff5c9c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:24:41.467085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc636e-4848-4e18-aecc-31e65ff5c9c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:41.467368 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc636e-4848-4e18-aecc-31e65ff5c9c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:41.467644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc636e-4848-4e18-aecc-31e65ff5c9c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:41.469354 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-15bc636e-4848-4e18-aecc-31e65ff5c9c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:41.534725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:24:41.612642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc636e-4848-4e18-aecc-31e65ff5c9c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:24:41.613788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc636e-4848-4e18-aecc-31e65ff5c9c0 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:24:41.634541 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 631/1311] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:24:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:24:43.903910 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46ba13-5446-4128-85ef-3df17dfb9a4d None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:43.904395 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46ba13-5446-4128-85ef-3df17dfb9a4d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:43.904518 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46ba13-5446-4128-85ef-3df17dfb9a4d None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:43.905875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 681/1312] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:24:43.911090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c0c84-e075-45db-8e4e-4add34e1a6fc None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:43.911529 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c0c84-e075-45db-8e4e-4add34e1a6fc None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:43.911625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c0c84-e075-45db-8e4e-4add34e1a6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:43.981918 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c66c0c84-e075-45db-8e4e-4add34e1a6fc None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:24:43.983144 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c66c0c84-e075-45db-8e4e-4add34e1a6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:44.080886 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 632/1313] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:24:44.366324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46ba13-5446-4128-85ef-3df17dfb9a4d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:24:44.367902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0f46ba13-5446-4128-85ef-3df17dfb9a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:44.471780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46ba13-5446-4128-85ef-3df17dfb9a4d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:24:44.544098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:24:44.546777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bc7c8c-6459-40d6-a0ca-d6ac09c2e29b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:24:44.547700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc7c8c-6459-40d6-a0ca-d6ac09c2e29b service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:44.548187 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc7c8c-6459-40d6-a0ca-d6ac09c2e29b service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:44.548476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bc7c8c-6459-40d6-a0ca-d6ac09c2e29b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:44.550459 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-26bc7c8c-6459-40d6-a0ca-d6ac09c2e29b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:44.673474 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc7c8c-6459-40d6-a0ca-d6ac09c2e29b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:24:44.675775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bc7c8c-6459-40d6-a0ca-d6ac09c2e29b service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:24:44.695911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 682/1314] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:24:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:24:46.967504 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19968910-3e67-4d42-a819-f5202d5a5725 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:46.968100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19968910-3e67-4d42-a819-f5202d5a5725 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:46.968199 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19968910-3e67-4d42-a819-f5202d5a5725 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:46.969566 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 633/1315] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:24:46.975202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e70f1-14a6-40fa-ac3f-b9c19e4e039e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:46.975609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e70f1-14a6-40fa-ac3f-b9c19e4e039e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:46.975766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e70f1-14a6-40fa-ac3f-b9c19e4e039e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:47.042450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-968e70f1-14a6-40fa-ac3f-b9c19e4e039e None None] MFA Rules not processed for user `0c802f73a7a64d1b862c3589c9d42f5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:24:47.043640 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-968e70f1-14a6-40fa-ac3f-b9c19e4e039e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:47.099387 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:24:47.146770 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 683/1316] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:24:46 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:24:47.434083 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19968910-3e67-4d42-a819-f5202d5a5725 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:24:47.435597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-19968910-3e67-4d42-a819-f5202d5a5725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:47.445785 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:24:47.536413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19968910-3e67-4d42-a819-f5202d5a5725 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:24:47.607292 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf878a-8810-43b0-8a7e-02b8672e8c85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:24:47.607818 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf878a-8810-43b0-8a7e-02b8672e8c85 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:47.608125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf878a-8810-43b0-8a7e-02b8672e8c85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:47.608395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf878a-8810-43b0-8a7e-02b8672e8c85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:47.610013 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-fadf878a-8810-43b0-8a7e-02b8672e8c85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:47.672295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:24:47.718220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf878a-8810-43b0-8a7e-02b8672e8c85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:24:47.719663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf878a-8810-43b0-8a7e-02b8672e8c85 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:24:47.738353 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 634/1317] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:24:47 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:24:50.221845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecdb15-44b4-47e0-8df1-77ee2a84b4a1 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:50.222378 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecdb15-44b4-47e0-8df1-77ee2a84b4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:50.222378 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecdb15-44b4-47e0-8df1-77ee2a84b4a1 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:50.223672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 684/1318] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:24:50.228888 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a0a5f-f457-4ab5-b8d6-6247689b99ca None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:50.229309 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a0a5f-f457-4ab5-b8d6-6247689b99ca None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:50.229452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a0a5f-f457-4ab5-b8d6-6247689b99ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:50.294827 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-071a0a5f-f457-4ab5-b8d6-6247689b99ca None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:24:50.296077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-071a0a5f-f457-4ab5-b8d6-6247689b99ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:50.391465 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 635/1319] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:24:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:24:50.693755 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f235626-fc67-4ba1-a047-cd019813f15a None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:50.694040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f235626-fc67-4ba1-a047-cd019813f15a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:50.694316 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f235626-fc67-4ba1-a047-cd019813f15a None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:50.695683 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 685/1320] 10.222.0.18 () {54 vars in 965 bytes} [Mon Aug 29 20:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:24:50.701603 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2539322-1fc6-4616-8d86-5c778fa726b7 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:50.701889 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2539322-1fc6-4616-8d86-5c778fa726b7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:50.702212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2539322-1fc6-4616-8d86-5c778fa726b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:50.723812 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:24:50.775955 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d2539322-1fc6-4616-8d86-5c778fa726b7 None None] MFA Rules not processed for user `d51e331a4c9c4f038bdb80f8c5b74e42`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:24:50.777163 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d2539322-1fc6-4616-8d86-5c778fa726b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:50.851424 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 636/1321] 10.222.0.18 () {58 vars in 1062 bytes} [Mon Aug 29 20:24:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:24:50.860321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f235626-fc67-4ba1-a047-cd019813f15a None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:24:50.861856 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2f235626-fc67-4ba1-a047-cd019813f15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:50.979799 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f235626-fc67-4ba1-a047-cd019813f15a None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:24:51.054038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732dc2f-b7bc-4fec-9a5c-365ae1a45d34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:24:51.054692 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732dc2f-b7bc-4fec-9a5c-365ae1a45d34 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:51.055089 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732dc2f-b7bc-4fec-9a5c-365ae1a45d34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:51.055409 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732dc2f-b7bc-4fec-9a5c-365ae1a45d34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:51.057117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5732dc2f-b7bc-4fec-9a5c-365ae1a45d34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:51.064950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:24:51.191318 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732dc2f-b7bc-4fec-9a5c-365ae1a45d34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:24:51.193670 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732dc2f-b7bc-4fec-9a5c-365ae1a45d34 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:24:51.217975 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 686/1322] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:24:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:24:54.279462 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2899a-ce14-4dee-86eb-66292f5a1ced None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:54.280052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2899a-ce14-4dee-86eb-66292f5a1ced None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:54.280052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2899a-ce14-4dee-86eb-66292f5a1ced None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:54.281400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 637/1323] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:24:54.286692 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16628be2-10ca-41b5-914f-0692d6429211 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:54.287116 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16628be2-10ca-41b5-914f-0692d6429211 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:54.287258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16628be2-10ca-41b5-914f-0692d6429211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:54.354807 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-16628be2-10ca-41b5-914f-0692d6429211 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:24:54.356036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-16628be2-10ca-41b5-914f-0692d6429211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:54.413024 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:24:54.452461 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 687/1324] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:24:54.740473 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2899a-ce14-4dee-86eb-66292f5a1ced None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:24:54.741964 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5ae2899a-ce14-4dee-86eb-66292f5a1ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:54.849609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2899a-ce14-4dee-86eb-66292f5a1ced None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:24:54.862722 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:24:54.926299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa9ffe-aab8-4639-81a1-2b054179a170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:24:54.926801 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa9ffe-aab8-4639-81a1-2b054179a170 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:54.927078 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa9ffe-aab8-4639-81a1-2b054179a170 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:54.927346 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa9ffe-aab8-4639-81a1-2b054179a170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:54.928968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-46fa9ffe-aab8-4639-81a1-2b054179a170 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:55.038451 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa9ffe-aab8-4639-81a1-2b054179a170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:24:55.040148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa9ffe-aab8-4639-81a1-2b054179a170 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:24:55.058808 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 638/1325] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:24:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:24:57.100573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fce78-e0f8-4fe9-8827-d55eddc152e2 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:57.101109 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fce78-e0f8-4fe9-8827-d55eddc152e2 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:57.101109 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fce78-e0f8-4fe9-8827-d55eddc152e2 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:57.102481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 688/1326] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:24:57.107612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db986c43-1d2b-4447-ab0c-d2f1a13c1733 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:57.108039 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db986c43-1d2b-4447-ab0c-d2f1a13c1733 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:57.108203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db986c43-1d2b-4447-ab0c-d2f1a13c1733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:57.156475 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:24:57.176626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-db986c43-1d2b-4447-ab0c-d2f1a13c1733 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:24:57.177858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-db986c43-1d2b-4447-ab0c-d2f1a13c1733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:57.287395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 639/1327] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:24:57.572461 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43fce78-e0f8-4fe9-8827-d55eddc152e2 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:24:57.574006 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d43fce78-e0f8-4fe9-8827-d55eddc152e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:57.694954 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43fce78-e0f8-4fe9-8827-d55eddc152e2 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:24:57.770043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b92545-4953-406f-9aa1-579a589c169b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:24:57.770662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b92545-4953-406f-9aa1-579a589c169b service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:57.770945 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b92545-4953-406f-9aa1-579a589c169b service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:57.771252 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b92545-4953-406f-9aa1-579a589c169b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:57.772892 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f5b92545-4953-406f-9aa1-579a589c169b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:24:57.790496 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:24:57.882478 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b92545-4953-406f-9aa1-579a589c169b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:24:57.883978 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b92545-4953-406f-9aa1-579a589c169b service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:24:57.902246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 689/1328] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:24:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:24:59.914560 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5963c-777d-40c7-a093-216f37a9f658 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:59.915081 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5963c-777d-40c7-a093-216f37a9f658 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:59.915174 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5963c-777d-40c7-a093-216f37a9f658 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:59.916511 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 640/1329] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:24:59.921618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0433d36-4fc6-472f-a5d0-99243986f5b7 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:24:59.922045 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0433d36-4fc6-472f-a5d0-99243986f5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:24:59.922187 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0433d36-4fc6-472f-a5d0-99243986f5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:24:59.991523 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d0433d36-4fc6-472f-a5d0-99243986f5b7 None None] MFA Rules not processed for user `0c802f73a7a64d1b862c3589c9d42f5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:24:59.992766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d0433d36-4fc6-472f-a5d0-99243986f5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:00.058685 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:00.092235 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 690/1330] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:24:59 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:00.375888 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5963c-777d-40c7-a093-216f37a9f658 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:00.378404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b6c5963c-777d-40c7-a093-216f37a9f658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:00.491102 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5963c-777d-40c7-a093-216f37a9f658 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:00.513983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:00.577004 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da27a5f-a64a-4873-bdaa-85a70d28064f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:00.577530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da27a5f-a64a-4873-bdaa-85a70d28064f service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:00.577803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da27a5f-a64a-4873-bdaa-85a70d28064f service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:00.578069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da27a5f-a64a-4873-bdaa-85a70d28064f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:00.579647 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8da27a5f-a64a-4873-bdaa-85a70d28064f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:00.701417 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da27a5f-a64a-4873-bdaa-85a70d28064f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:00.702847 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da27a5f-a64a-4873-bdaa-85a70d28064f service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:00.721561 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 641/1331] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:00 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:02.896383 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5377083-94ac-4baf-a0f7-3c55c8c5af99 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:02.896976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5377083-94ac-4baf-a0f7-3c55c8c5af99 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:02.896976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5377083-94ac-4baf-a0f7-3c55c8c5af99 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:02.898424 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 691/1332] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:25:02.903681 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46652e-4db2-46e4-a35e-a00c244aab62 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:02.904132 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46652e-4db2-46e4-a35e-a00c244aab62 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:02.904288 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd46652e-4db2-46e4-a35e-a00c244aab62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:02.965935 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:02.968748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-dd46652e-4db2-46e4-a35e-a00c244aab62 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:02.970519 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-dd46652e-4db2-46e4-a35e-a00c244aab62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:03.066417 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 642/1333] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:03.351249 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5377083-94ac-4baf-a0f7-3c55c8c5af99 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:03.352816 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a5377083-94ac-4baf-a0f7-3c55c8c5af99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:03.456440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5377083-94ac-4baf-a0f7-3c55c8c5af99 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:03.533550 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d46fc3-0725-487e-9add-574fb7085692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:03.534167 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d46fc3-0725-487e-9add-574fb7085692 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:03.534595 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d46fc3-0725-487e-9add-574fb7085692 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:03.534877 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d46fc3-0725-487e-9add-574fb7085692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:03.536496 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a1d46fc3-0725-487e-9add-574fb7085692 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:03.565258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:03.649459 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d46fc3-0725-487e-9add-574fb7085692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:03.650906 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d46fc3-0725-487e-9add-574fb7085692 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:03.668963 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 692/1334] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:05.643283 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b175f9-7249-4445-baa5-093b50358e71 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:05.643821 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b175f9-7249-4445-baa5-093b50358e71 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:05.643821 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b175f9-7249-4445-baa5-093b50358e71 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:05.645277 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 643/1335] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:25:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:05.651015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ee830-1f61-4954-930e-c7ac2ef373b0 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:05.651550 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ee830-1f61-4954-930e-c7ac2ef373b0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:05.651550 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ee830-1f61-4954-930e-c7ac2ef373b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:05.718768 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-5a7ee830-1f61-4954-930e-c7ac2ef373b0 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:05.720015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5a7ee830-1f61-4954-930e-c7ac2ef373b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:05.796744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:05.820859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 693/1336] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:06.104962 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b175f9-7249-4445-baa5-093b50358e71 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:06.106509 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e8b175f9-7249-4445-baa5-093b50358e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:06.208122 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b175f9-7249-4445-baa5-093b50358e71 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:06.234997 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:06.279131 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07459a5-f31b-49fd-9fb9-4f159400f641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:06.279741 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07459a5-f31b-49fd-9fb9-4f159400f641 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:06.280056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07459a5-f31b-49fd-9fb9-4f159400f641 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:06.280333 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07459a5-f31b-49fd-9fb9-4f159400f641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:06.281922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c07459a5-f31b-49fd-9fb9-4f159400f641 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:06.390150 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07459a5-f31b-49fd-9fb9-4f159400f641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:06.391578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07459a5-f31b-49fd-9fb9-4f159400f641 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:06.409810 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 644/1337] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:08.650263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b1660-9ed5-4719-8ce4-c319866982a4 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:08.650804 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b1660-9ed5-4719-8ce4-c319866982a4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:08.650804 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b1660-9ed5-4719-8ce4-c319866982a4 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:08.652324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 694/1338] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:08.659116 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f899ed-988f-4a8d-a99f-19345d8cd035 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:08.659517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f899ed-988f-4a8d-a99f-19345d8cd035 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:08.659673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f899ed-988f-4a8d-a99f-19345d8cd035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:08.739438 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-27f899ed-988f-4a8d-a99f-19345d8cd035 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:08.740851 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-27f899ed-988f-4a8d-a99f-19345d8cd035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:08.749424 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:08.844195 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 645/1339] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:25:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:09.128220 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b1660-9ed5-4719-8ce4-c319866982a4 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:09.129862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a62b1660-9ed5-4719-8ce4-c319866982a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:09.230543 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b1660-9ed5-4719-8ce4-c319866982a4 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:09.302117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fe990-b23a-44d9-b2b7-d1b3c6fdda7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:09.302728 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fe990-b23a-44d9-b2b7-d1b3c6fdda7f service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:09.303006 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fe990-b23a-44d9-b2b7-d1b3c6fdda7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:09.303278 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fe990-b23a-44d9-b2b7-d1b3c6fdda7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:09.305062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4a3fe990-b23a-44d9-b2b7-d1b3c6fdda7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:09.340213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:09.412196 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fe990-b23a-44d9-b2b7-d1b3c6fdda7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:09.413825 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fe990-b23a-44d9-b2b7-d1b3c6fdda7f service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:09.431245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 695/1340] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:13.457919 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86311d-c251-4330-9127-9eec884829c3 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:13.458657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86311d-c251-4330-9127-9eec884829c3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:13.458657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86311d-c251-4330-9127-9eec884829c3 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:13.459815 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 646/1341] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:13.465180 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f5c92-b050-47f5-af29-299b18b35b92 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:13.465599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f5c92-b050-47f5-af29-299b18b35b92 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:13.465715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f5c92-b050-47f5-af29-299b18b35b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:13.534629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-c64f5c92-b050-47f5-af29-299b18b35b92 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:13.535836 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c64f5c92-b050-47f5-af29-299b18b35b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:13.629621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:13.641143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 696/1342] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:13.926273 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86311d-c251-4330-9127-9eec884829c3 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:13.927856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-cb86311d-c251-4330-9127-9eec884829c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:14.036153 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86311d-c251-4330-9127-9eec884829c3 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:14.074856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:14.113256 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe08e88-bf3b-455a-a470-7b5fc880be61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:14.113889 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe08e88-bf3b-455a-a470-7b5fc880be61 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:14.114189 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe08e88-bf3b-455a-a470-7b5fc880be61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:14.114532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe08e88-bf3b-455a-a470-7b5fc880be61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:14.116225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-cbe08e88-bf3b-455a-a470-7b5fc880be61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:14.239281 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe08e88-bf3b-455a-a470-7b5fc880be61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:14.240670 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe08e88-bf3b-455a-a470-7b5fc880be61 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:14.258927 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 647/1343] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:16.967633 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157e162-1c38-4a85-a755-7fbeb4bc8523 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:16.968471 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157e162-1c38-4a85-a755-7fbeb4bc8523 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:16.968471 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157e162-1c38-4a85-a755-7fbeb4bc8523 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:16.969764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 697/1344] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:16.975781 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021e63a-7b9c-4131-b8fd-b4ea1a2a9062 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:16.976331 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021e63a-7b9c-4131-b8fd-b4ea1a2a9062 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:16.976424 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021e63a-7b9c-4131-b8fd-b4ea1a2a9062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:17.057703 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5021e63a-7b9c-4131-b8fd-b4ea1a2a9062 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:17.058933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5021e63a-7b9c-4131-b8fd-b4ea1a2a9062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:17.079149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:17.163403 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 648/1345] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:25:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:17.452613 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157e162-1c38-4a85-a755-7fbeb4bc8523 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:17.454142 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9157e162-1c38-4a85-a755-7fbeb4bc8523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:17.564995 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157e162-1c38-4a85-a755-7fbeb4bc8523 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:17.643051 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f86f8-4ff7-4c0d-9caf-15b0150b438d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:17.643687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f86f8-4ff7-4c0d-9caf-15b0150b438d service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:17.643996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f86f8-4ff7-4c0d-9caf-15b0150b438d service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:17.644276 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f86f8-4ff7-4c0d-9caf-15b0150b438d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:17.645891 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-080f86f8-4ff7-4c0d-9caf-15b0150b438d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:17.696820 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:17.781110 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f86f8-4ff7-4c0d-9caf-15b0150b438d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:17.782562 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f86f8-4ff7-4c0d-9caf-15b0150b438d service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:17.803440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 698/1346] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:21.856446 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2582d3-6a7f-41de-9203-02770ec87503 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:21.856997 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2582d3-6a7f-41de-9203-02770ec87503 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:21.856997 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2582d3-6a7f-41de-9203-02770ec87503 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:21.858974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 649/1347] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:25:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:21.864569 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12872d-c64c-4c12-a39d-48da0c596cff None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:21.865080 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12872d-c64c-4c12-a39d-48da0c596cff None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:21.865178 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12872d-c64c-4c12-a39d-48da0c596cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:21.965572 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-db12872d-c64c-4c12-a39d-48da0c596cff None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:21.966694 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-db12872d-c64c-4c12-a39d-48da0c596cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:22.093769 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:22.097455 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 699/1348] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:22.378811 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2582d3-6a7f-41de-9203-02770ec87503 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:22.380407 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-cd2582d3-6a7f-41de-9203-02770ec87503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:22.495992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2582d3-6a7f-41de-9203-02770ec87503 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:22.553872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:22.591578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1952f21-2b89-4657-a697-29dd929beda8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:22.592321 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1952f21-2b89-4657-a697-29dd929beda8 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:22.592603 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1952f21-2b89-4657-a697-29dd929beda8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:22.592877 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1952f21-2b89-4657-a697-29dd929beda8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:22.594775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a1952f21-2b89-4657-a697-29dd929beda8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:22.744975 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1952f21-2b89-4657-a697-29dd929beda8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:22.746463 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1952f21-2b89-4657-a697-29dd929beda8 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:22.764865 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 650/1349] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:27.728964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d4fe1-1922-42f0-8b0b-9d16e190106d None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:27.729439 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d4fe1-1922-42f0-8b0b-9d16e190106d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:27.729718 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7d4fe1-1922-42f0-8b0b-9d16e190106d None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:27.733503 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 700/1350] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 29 20:25:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:27.740269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66438fd6-f2bc-4eef-a9b9-a2a9cb5b0cd9 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:27.740791 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66438fd6-f2bc-4eef-a9b9-a2a9cb5b0cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:27.740986 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66438fd6-f2bc-4eef-a9b9-a2a9cb5b0cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:27.843604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-66438fd6-f2bc-4eef-a9b9-a2a9cb5b0cd9 None None] MFA Rules not processed for user `fa0e659eee5e4a9da8d3d7337cb73798`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:27.846004 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-66438fd6-f2bc-4eef-a9b9-a2a9cb5b0cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:27.871094 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:27.988671 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 651/1351] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 29 20:25:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:32.605477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77682908-da40-4464-84cb-d7ba3b332666 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:32.606034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77682908-da40-4464-84cb-d7ba3b332666 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:32.606034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77682908-da40-4464-84cb-d7ba3b332666 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:32.607501 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 701/1352] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 29 20:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:32.612332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5907a-6a4f-4570-a4a3-d6f32e7d0a1c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:32.612782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5907a-6a4f-4570-a4a3-d6f32e7d0a1c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:32.612887 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5907a-6a4f-4570-a4a3-d6f32e7d0a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:32.698752 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-dda5907a-6a4f-4570-a4a3-d6f32e7d0a1c None None] MFA Rules not processed for user `fa0e659eee5e4a9da8d3d7337cb73798`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:32.700454 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-dda5907a-6a4f-4570-a4a3-d6f32e7d0a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:32.780883 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:32.821217 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 652/1353] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 29 20:25:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:25:34.413407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbe2c0-43e6-47cd-a0fb-f1a808c03399 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:34.413969 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbe2c0-43e6-47cd-a0fb-f1a808c03399 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:34.413969 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abbe2c0-43e6-47cd-a0fb-f1a808c03399 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:34.415365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 702/1354] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 29 20:25:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:34.421111 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1e786-776a-476a-85f6-02103ef07ffe None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:34.421711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1e786-776a-476a-85f6-02103ef07ffe None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:34.421711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1e786-776a-476a-85f6-02103ef07ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:34.503324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-9de1e786-776a-476a-85f6-02103ef07ffe None None] MFA Rules not processed for user `fa0e659eee5e4a9da8d3d7337cb73798`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:34.504570 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9de1e786-776a-476a-85f6-02103ef07ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:34.614091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 653/1355] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 29 20:25:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:37.765936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6e182-919e-4232-8086-e33fd1e66ee8 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:37.766475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6e182-919e-4232-8086-e33fd1e66ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:37.766475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6e182-919e-4232-8086-e33fd1e66ee8 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:37.770900 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 703/1356] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:25:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:37.774838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230a726-699e-4a53-a74a-ce4bfebc86ca None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:37.775295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230a726-699e-4a53-a74a-ce4bfebc86ca None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:37.775393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6230a726-699e-4a53-a74a-ce4bfebc86ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:37.815373 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:37.849190 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6230a726-699e-4a53-a74a-ce4bfebc86ca None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:37.851252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6230a726-699e-4a53-a74a-ce4bfebc86ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:37.971566 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 654/1357] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:38.282087 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2a556-ccef-42ae-b97a-8d8ac9b6ed26 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:38.283437 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2a556-ccef-42ae-b97a-8d8ac9b6ed26 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:38.285484 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2a556-ccef-42ae-b97a-8d8ac9b6ed26 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:38.291033 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 704/1358] 10.222.0.18 () {54 vars in 962 bytes} [Mon Aug 29 20:25:38 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:38.302596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efdf0e-24f9-448b-a949-5549ac3bbe87 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:38.303360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efdf0e-24f9-448b-a949-5549ac3bbe87 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:38.304095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8efdf0e-24f9-448b-a949-5549ac3bbe87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:38.409812 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-a8efdf0e-24f9-448b-a949-5549ac3bbe87 None None] MFA Rules not processed for user `c0c44754a1c04a2b87081aba6cb509d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:38.411010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a8efdf0e-24f9-448b-a949-5549ac3bbe87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:38.466239 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:38.542252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 655/1359] 10.222.0.18 () {58 vars in 1059 bytes} [Mon Aug 29 20:25:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:38.550977 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2a556-ccef-42ae-b97a-8d8ac9b6ed26 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:38.552522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-bec2a556-ccef-42ae-b97a-8d8ac9b6ed26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:38.702254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2a556-ccef-42ae-b97a-8d8ac9b6ed26 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:38.778454 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:38.821611 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d8fe0e-7fd9-44fb-8422-135fddd91f5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:38.822141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8fe0e-7fd9-44fb-8422-135fddd91f5c service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:38.822419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8fe0e-7fd9-44fb-8422-135fddd91f5c service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:38.822691 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d8fe0e-7fd9-44fb-8422-135fddd91f5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:38.824308 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-52d8fe0e-7fd9-44fb-8422-135fddd91f5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:38.921788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8fe0e-7fd9-44fb-8422-135fddd91f5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:38.923424 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d8fe0e-7fd9-44fb-8422-135fddd91f5c service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:38.944169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 705/1360] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:25:41.110551 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b7330-8635-4ac2-93a1-8c2fc097055a None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:41.111139 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b7330-8635-4ac2-93a1-8c2fc097055a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:41.111139 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b7330-8635-4ac2-93a1-8c2fc097055a None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:41.113265 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 656/1361] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:41.117727 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e53d4-3bd5-45a8-9303-80e3cc5d8d62 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:41.118119 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e53d4-3bd5-45a8-9303-80e3cc5d8d62 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:41.118333 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e53d4-3bd5-45a8-9303-80e3cc5d8d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:41.190505 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-3c4e53d4-3bd5-45a8-9303-80e3cc5d8d62 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:41.191730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3c4e53d4-3bd5-45a8-9303-80e3cc5d8d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:41.231030 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:41.308250 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 706/1362] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:41.597586 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b7330-8635-4ac2-93a1-8c2fc097055a None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:41.597586 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c61b7330-8635-4ac2-93a1-8c2fc097055a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:41.767438 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b7330-8635-4ac2-93a1-8c2fc097055a None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:41.777965 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:41.899725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe63d1f-6e9e-4e8b-8c06-53940a7e583e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:41.900365 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe63d1f-6e9e-4e8b-8c06-53940a7e583e service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:41.900649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe63d1f-6e9e-4e8b-8c06-53940a7e583e service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:41.900954 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe63d1f-6e9e-4e8b-8c06-53940a7e583e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:41.902701 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-efe63d1f-6e9e-4e8b-8c06-53940a7e583e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:42.023886 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe63d1f-6e9e-4e8b-8c06-53940a7e583e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:42.026205 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe63d1f-6e9e-4e8b-8c06-53940a7e583e service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:42.049898 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 657/1363] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:44.324239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262e269-b9b1-488e-8e89-ed00b1702d77 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:44.324788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262e269-b9b1-488e-8e89-ed00b1702d77 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:44.324888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262e269-b9b1-488e-8e89-ed00b1702d77 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:44.326432 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 707/1364] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:44.332090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882d880-44c5-4227-b873-3aa46f0d17cb None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:44.332562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882d880-44c5-4227-b873-3aa46f0d17cb None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:44.332685 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882d880-44c5-4227-b873-3aa46f0d17cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:44.352558 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:44.417538 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-0882d880-44c5-4227-b873-3aa46f0d17cb None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:44.418923 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0882d880-44c5-4227-b873-3aa46f0d17cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:44.526452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 658/1365] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:44.814978 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6be8c-fad0-48b7-8221-f82f40a10ea8 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:44.815505 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6be8c-fad0-48b7-8221-f82f40a10ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:44.815781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6be8c-fad0-48b7-8221-f82f40a10ea8 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:44.817149 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 708/1366] 10.222.0.18 () {54 vars in 962 bytes} [Mon Aug 29 20:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:44.822827 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57db933-f122-4b57-9146-f4a0874ca7f4 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:44.823107 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57db933-f122-4b57-9146-f4a0874ca7f4 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:44.823377 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57db933-f122-4b57-9146-f4a0874ca7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:44.910098 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b57db933-f122-4b57-9146-f4a0874ca7f4 None None] MFA Rules not processed for user `c0c44754a1c04a2b87081aba6cb509d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:44.911364 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b57db933-f122-4b57-9146-f4a0874ca7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:44.954619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:45.045689 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 659/1367] 10.222.0.18 () {58 vars in 1059 bytes} [Mon Aug 29 20:25:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:45.053916 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6be8c-fad0-48b7-8221-f82f40a10ea8 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:45.055537 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c1e6be8c-fad0-48b7-8221-f82f40a10ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:45.214930 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6be8c-fad0-48b7-8221-f82f40a10ea8 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:45.236334 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:45.343840 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60d099-e3d7-481a-a517-f180cd0ee13e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:45.344553 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60d099-e3d7-481a-a517-f180cd0ee13e service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:45.344829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60d099-e3d7-481a-a517-f180cd0ee13e service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:45.345110 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60d099-e3d7-481a-a517-f180cd0ee13e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:45.346810 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5d60d099-e3d7-481a-a517-f180cd0ee13e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:45.444136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60d099-e3d7-481a-a517-f180cd0ee13e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:45.445754 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d60d099-e3d7-481a-a517-f180cd0ee13e service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:45.464726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 709/1368] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:46.244139 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbdf2b-f893-414a-9269-f2762d78b4d5 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:46.244702 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbdf2b-f893-414a-9269-f2762d78b4d5 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:46.244896 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbdf2b-f893-414a-9269-f2762d78b4d5 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:46.246521 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 660/1369] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:46.251751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e189f-5d9b-4e12-97d7-301d8f3bab9c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:46.252272 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e189f-5d9b-4e12-97d7-301d8f3bab9c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:46.252680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e189f-5d9b-4e12-97d7-301d8f3bab9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:46.292023 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:46.329419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-7b4e189f-5d9b-4e12-97d7-301d8f3bab9c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:46.330771 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7b4e189f-5d9b-4e12-97d7-301d8f3bab9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:46.440938 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 710/1370] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:46.988807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec5f46-2d36-4502-9343-4dd61480f74d None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:46.989095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec5f46-2d36-4502-9343-4dd61480f74d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:46.989372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec5f46-2d36-4502-9343-4dd61480f74d None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:46.990876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 661/1371] 10.222.0.18 () {54 vars in 964 bytes} [Mon Aug 29 20:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:46.996072 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff88bf-fcb5-4f66-a0d5-c7304aea1769 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:46.996363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff88bf-fcb5-4f66-a0d5-c7304aea1769 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:46.996642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff88bf-fcb5-4f66-a0d5-c7304aea1769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:47.080228 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-69ff88bf-fcb5-4f66-a0d5-c7304aea1769 None None] MFA Rules not processed for user `85624cea63af40fda09f57ddf2cd5c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:47.081460 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-69ff88bf-fcb5-4f66-a0d5-c7304aea1769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:47.135205 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:47.171400 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 711/1372] 10.222.0.18 () {58 vars in 1061 bytes} [Mon Aug 29 20:25:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:25:47.179491 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec5f46-2d36-4502-9343-4dd61480f74d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:47.181306 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-60ec5f46-2d36-4502-9343-4dd61480f74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:47.292697 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec5f46-2d36-4502-9343-4dd61480f74d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:47.335323 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:47.377575 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8abff3-2dd0-4996-904f-77b600a941a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:47.378194 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8abff3-2dd0-4996-904f-77b600a941a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:47.378517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8abff3-2dd0-4996-904f-77b600a941a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:47.378805 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8abff3-2dd0-4996-904f-77b600a941a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:47.380529 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-df8abff3-2dd0-4996-904f-77b600a941a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:47.472173 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f497b-f161-4942-bcd5-972973b391f5 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:47.472733 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f497b-f161-4942-bcd5-972973b391f5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:47.472733 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f497b-f161-4942-bcd5-972973b391f5 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:47.477939 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 712/1373] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:25:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:47.478934 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd455b-4d67-43bb-8049-2c73a974e54b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:47.479221 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd455b-4d67-43bb-8049-2c73a974e54b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:47.479513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd455b-4d67-43bb-8049-2c73a974e54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:47.516938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8abff3-2dd0-4996-904f-77b600a941a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:47.518387 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8abff3-2dd0-4996-904f-77b600a941a5 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:47.538376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 662/1374] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:25:47.556917 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2acd455b-4d67-43bb-8049-2c73a974e54b None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:47.558412 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2acd455b-4d67-43bb-8049-2c73a974e54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:47.692553 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 713/1375] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:48.078185 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8abff3-2dd0-4996-904f-77b600a941a5 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:48.079910 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-df8abff3-2dd0-4996-904f-77b600a941a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:48.148468 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:48.180329 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8abff3-2dd0-4996-904f-77b600a941a5 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:48.315631 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce16f29-749d-4168-ae00-1d057ad59bbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:48.316366 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce16f29-749d-4168-ae00-1d057ad59bbc service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:48.316642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce16f29-749d-4168-ae00-1d057ad59bbc service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:48.316908 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce16f29-749d-4168-ae00-1d057ad59bbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:48.318490 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8ce16f29-749d-4168-ae00-1d057ad59bbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:48.402286 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:48.414225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce16f29-749d-4168-ae00-1d057ad59bbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:48.415858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce16f29-749d-4168-ae00-1d057ad59bbc service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:48.433948 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 663/1376] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:50.402561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502f0a1-203d-4263-88c6-18bf08613c3f None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:50.403083 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502f0a1-203d-4263-88c6-18bf08613c3f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:50.403179 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502f0a1-203d-4263-88c6-18bf08613c3f None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:50.404504 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 714/1377] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:50.409744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a460ae-3b60-4932-ab90-0b2eb4dc7d62 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:50.410155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a460ae-3b60-4932-ab90-0b2eb4dc7d62 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:50.410297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a460ae-3b60-4932-ab90-0b2eb4dc7d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:50.478022 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-54a460ae-3b60-4932-ab90-0b2eb4dc7d62 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:50.479284 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-54a460ae-3b60-4932-ab90-0b2eb4dc7d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:50.589421 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 664/1378] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:50.882989 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502f0a1-203d-4263-88c6-18bf08613c3f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:50.884544 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d502f0a1-203d-4263-88c6-18bf08613c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:50.945470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:51.040925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502f0a1-203d-4263-88c6-18bf08613c3f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:51.201649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b081b0a-ab47-472e-88f5-ddaebe6bcd73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:51.203247 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b081b0a-ab47-472e-88f5-ddaebe6bcd73 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:51.203632 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b081b0a-ab47-472e-88f5-ddaebe6bcd73 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:51.204692 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b081b0a-ab47-472e-88f5-ddaebe6bcd73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:51.206589 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6b081b0a-ab47-472e-88f5-ddaebe6bcd73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:51.222321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:51.330182 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b081b0a-ab47-472e-88f5-ddaebe6bcd73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:51.331903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b081b0a-ab47-472e-88f5-ddaebe6bcd73 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:51.351861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 715/1379] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:53.345179 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2beef-15e4-44d0-9381-6c2c21f87a24 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:53.345725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2beef-15e4-44d0-9381-6c2c21f87a24 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:53.345725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2beef-15e4-44d0-9381-6c2c21f87a24 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:53.347240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 665/1380] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:53.352561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a84368-4def-4b3d-9a32-800f0f4bddc2 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:53.352969 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a84368-4def-4b3d-9a32-800f0f4bddc2 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:53.353158 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a84368-4def-4b3d-9a32-800f0f4bddc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:53.424468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-e5a84368-4def-4b3d-9a32-800f0f4bddc2 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:53.425676 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e5a84368-4def-4b3d-9a32-800f0f4bddc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:53.524854 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:53.539193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 716/1381] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:25:53.829401 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2beef-15e4-44d0-9381-6c2c21f87a24 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:53.830905 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-97c2beef-15e4-44d0-9381-6c2c21f87a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:53.895848 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:53.986164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2beef-15e4-44d0-9381-6c2c21f87a24 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:54.151902 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3f561-e8b7-4ae9-b447-75b1cc90589d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:54.152647 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3f561-e8b7-4ae9-b447-75b1cc90589d service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:54.153147 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3f561-e8b7-4ae9-b447-75b1cc90589d service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:54.153582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3f561-e8b7-4ae9-b447-75b1cc90589d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:54.156023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5fc3f561-e8b7-4ae9-b447-75b1cc90589d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:54.188491 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:54.269908 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3f561-e8b7-4ae9-b447-75b1cc90589d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:54.271576 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3f561-e8b7-4ae9-b447-75b1cc90589d service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:54.293135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 666/1382] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:56.861355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e592684-4a25-49bb-89b8-b637780ed6df None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:56.861912 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e592684-4a25-49bb-89b8-b637780ed6df None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:56.861912 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e592684-4a25-49bb-89b8-b637780ed6df None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:56.863260 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 717/1383] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:25:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:56.870201 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62081dd-2fc7-40f4-8318-aa052705ad0f None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:56.870661 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62081dd-2fc7-40f4-8318-aa052705ad0f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:56.870757 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62081dd-2fc7-40f4-8318-aa052705ad0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:56.948856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d62081dd-2fc7-40f4-8318-aa052705ad0f None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:56.950091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d62081dd-2fc7-40f4-8318-aa052705ad0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:57.070238 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:57.073651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 667/1384] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:25:57.372435 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e592684-4a25-49bb-89b8-b637780ed6df None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:25:57.373951 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5e592684-4a25-49bb-89b8-b637780ed6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:57.540867 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e592684-4a25-49bb-89b8-b637780ed6df None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:25:57.619575 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:25:57.694571 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e277a88-ba4d-4f47-a20d-e6a41a9cf010 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:25:57.695554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e277a88-ba4d-4f47-a20d-e6a41a9cf010 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:57.696096 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e277a88-ba4d-4f47-a20d-e6a41a9cf010 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:57.696591 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e277a88-ba4d-4f47-a20d-e6a41a9cf010 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:57.700050 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6e277a88-ba4d-4f47-a20d-e6a41a9cf010 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:57.824620 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e277a88-ba4d-4f47-a20d-e6a41a9cf010 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:25:57.826958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e277a88-ba4d-4f47-a20d-e6a41a9cf010 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:25:57.852656 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 718/1385] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:25:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:25:59.811119 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b6715-91fa-4bdc-a4e1-2234fd5afe25 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:59.811802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b6715-91fa-4bdc-a4e1-2234fd5afe25 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:59.811802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b6715-91fa-4bdc-a4e1-2234fd5afe25 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:59.813497 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 668/1386] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:25:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:25:59.821883 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7cce3-8207-4a9e-84ae-3ef557afa06a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:25:59.822368 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7cce3-8207-4a9e-84ae-3ef557afa06a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:25:59.822595 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7cce3-8207-4a9e-84ae-3ef557afa06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:25:59.902737 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-12f7cce3-8207-4a9e-84ae-3ef557afa06a None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:25:59.904081 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-12f7cce3-8207-4a9e-84ae-3ef557afa06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:25:59.935427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:00.027179 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 719/1387] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:00.341262 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b6715-91fa-4bdc-a4e1-2234fd5afe25 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:00.342836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d67b6715-91fa-4bdc-a4e1-2234fd5afe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:00.515238 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b6715-91fa-4bdc-a4e1-2234fd5afe25 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:00.608234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:00.690440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752959ed-1ad6-4ed0-a191-6dd512463f92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:00.691281 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752959ed-1ad6-4ed0-a191-6dd512463f92 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:00.691648 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752959ed-1ad6-4ed0-a191-6dd512463f92 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:00.692034 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752959ed-1ad6-4ed0-a191-6dd512463f92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:00.694296 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-752959ed-1ad6-4ed0-a191-6dd512463f92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:00.793613 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752959ed-1ad6-4ed0-a191-6dd512463f92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:00.795213 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752959ed-1ad6-4ed0-a191-6dd512463f92 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:00.813894 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 669/1388] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:02.829497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8adf5c-4c13-4018-b376-491e498586cc None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:02.830037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8adf5c-4c13-4018-b376-491e498586cc None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:02.830037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8adf5c-4c13-4018-b376-491e498586cc None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:02.831400 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 720/1389] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:02.836833 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c2682-a397-4101-944a-5f732d0858e1 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:02.837524 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c2682-a397-4101-944a-5f732d0858e1 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:02.837524 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c2682-a397-4101-944a-5f732d0858e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:02.922768 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff61128-87c5-4beb-9d56-72db0f102a3c None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:02.923091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff61128-87c5-4beb-9d56-72db0f102a3c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:02.923320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff61128-87c5-4beb-9d56-72db0f102a3c None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:02.924692 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 721/1390] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:26:02.926348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ee3c2682-a397-4101-944a-5f732d0858e1 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:02.927548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ee3c2682-a397-4101-944a-5f732d0858e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:02.929846 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398fe3e-de1b-4272-bcdf-50e4f636de5e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:02.930596 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398fe3e-de1b-4272-bcdf-50e4f636de5e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:02.930596 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398fe3e-de1b-4272-bcdf-50e4f636de5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:02.968028 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:02.998970 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a398fe3e-de1b-4272-bcdf-50e4f636de5e None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:03.000367 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a398fe3e-de1b-4272-bcdf-50e4f636de5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:03.034571 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 670/1391] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:03.107662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 722/1392] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:26:03.324419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c2682-a397-4101-944a-5f732d0858e1 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:03.325642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ee3c2682-a397-4101-944a-5f732d0858e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:03.419528 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3c96d-8cda-4ee9-981c-615747f4bede None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:03.419813 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3c96d-8cda-4ee9-981c-615747f4bede None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:03.420131 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3c96d-8cda-4ee9-981c-615747f4bede None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:03.421509 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 723/1393] 10.222.0.18 () {54 vars in 967 bytes} [Mon Aug 29 20:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:26:03.427170 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c84ee-6179-476e-adbe-315c45a0d16e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:03.427457 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c84ee-6179-476e-adbe-315c45a0d16e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:03.427733 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c84ee-6179-476e-adbe-315c45a0d16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:03.437787 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:03.468049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c2682-a397-4101-944a-5f732d0858e1 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:03.521235 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-3f7c84ee-6179-476e-adbe-315c45a0d16e None None] MFA Rules not processed for user `5d022ed8d2ac43ae816aa396a8e2f17a`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:03.522651 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3f7c84ee-6179-476e-adbe-315c45a0d16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:03.532090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:03.605724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013bb229-e1d4-43a8-b979-951ebe811c33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:03.606346 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bb229-e1d4-43a8-b979-951ebe811c33 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:03.606625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bb229-e1d4-43a8-b979-951ebe811c33 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:03.606895 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bb229-e1d4-43a8-b979-951ebe811c33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:03.608569 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-013bb229-e1d4-43a8-b979-951ebe811c33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:03.610210 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 724/1394] 10.222.0.18 () {58 vars in 1064 bytes} [Mon Aug 29 20:26:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:26:03.617981 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c84ee-6179-476e-adbe-315c45a0d16e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:03.628061 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c84ee-6179-476e-adbe-315c45a0d16e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:03.705673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013bb229-e1d4-43a8-b979-951ebe811c33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:03.707195 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013bb229-e1d4-43a8-b979-951ebe811c33 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:03.709301 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1db6f-06c3-4dcb-81dd-432abb621825 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d022ed8d2ac43ae816aa396a8e2f17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:03.709920 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1db6f-06c3-4dcb-81dd-432abb621825 service glance] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:03.710203 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1db6f-06c3-4dcb-81dd-432abb621825 service glance] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:03.710477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1db6f-06c3-4dcb-81dd-432abb621825 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:03.712560 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-36f1db6f-06c3-4dcb-81dd-432abb621825 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:03.725877 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 671/1395] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:03.745572 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:03.859098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1db6f-06c3-4dcb-81dd-432abb621825 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:03.861243 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1db6f-06c3-4dcb-81dd-432abb621825 service glance] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:03.883381 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 725/1396] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:04.674280 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d32bb-719b-483c-9eca-2b469abf0f7c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:04.674810 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d32bb-719b-483c-9eca-2b469abf0f7c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:04.674810 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d32bb-719b-483c-9eca-2b469abf0f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:04.749260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-856d32bb-719b-483c-9eca-2b469abf0f7c None None] MFA Rules not processed for user `61709ccc1e244185b252179451cd860f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:04.751006 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-856d32bb-719b-483c-9eca-2b469abf0f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:04.758946 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:04.856867 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 672/1397] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 29 20:26:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:04.903197 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9de745-e09f-496d-a47c-4929a63453b5 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:04.903711 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9de745-e09f-496d-a47c-4929a63453b5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:04.903796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9de745-e09f-496d-a47c-4929a63453b5 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:04.905250 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 726/1398] 10.222.0.18 () {54 vars in 943 bytes} [Mon Aug 29 20:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:04.911608 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5319d09-d6aa-437f-8b87-e30265b24af0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:04.911899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5319d09-d6aa-437f-8b87-e30265b24af0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:04.912212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5319d09-d6aa-437f-8b87-e30265b24af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:04.992486 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c5319d09-d6aa-437f-8b87-e30265b24af0 None None] MFA Rules not processed for user `599129b7d8ff47efb145787414ddb80d`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:04.993699 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c5319d09-d6aa-437f-8b87-e30265b24af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:05.075868 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:05.103483 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 673/1399] 10.222.0.18 () {58 vars in 1040 bytes} [Mon Aug 29 20:26:04 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:05.113525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9de745-e09f-496d-a47c-4929a63453b5 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:05.115112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ff9de745-e09f-496d-a47c-4929a63453b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:05.264440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:05.279113 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9de745-e09f-496d-a47c-4929a63453b5 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:05.410410 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f06534-c0b5-48e1-9151-8c5acae0664a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599129b7d8ff47efb145787414ddb80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:05.410946 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f06534-c0b5-48e1-9151-8c5acae0664a service swift] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:05.411229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f06534-c0b5-48e1-9151-8c5acae0664a service swift] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:05.411502 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f06534-c0b5-48e1-9151-8c5acae0664a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:05.413424 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-58f06534-c0b5-48e1-9151-8c5acae0664a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:05.511147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f06534-c0b5-48e1-9151-8c5acae0664a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:05.512798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f06534-c0b5-48e1-9151-8c5acae0664a service swift] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:05.522414 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 727/1400] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 29 20:26:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:26:05.714080 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1957f52-dfbf-4847-a4dd-d01f5e917155 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:05.714603 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1957f52-dfbf-4847-a4dd-d01f5e917155 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:05.714603 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1957f52-dfbf-4847-a4dd-d01f5e917155 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:05.716941 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 674/1401] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:05.721697 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e133446-c4fd-44ca-8145-b80c003c323c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:05.721988 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e133446-c4fd-44ca-8145-b80c003c323c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:05.722267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e133446-c4fd-44ca-8145-b80c003c323c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:05.761492 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:05.816655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-0e133446-c4fd-44ca-8145-b80c003c323c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:05.818153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0e133446-c4fd-44ca-8145-b80c003c323c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:05.931032 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 728/1402] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:26:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:06.220669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1957f52-dfbf-4847-a4dd-d01f5e917155 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:06.222380 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f1957f52-dfbf-4847-a4dd-d01f5e917155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:06.409575 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1957f52-dfbf-4847-a4dd-d01f5e917155 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:06.478636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:06.548467 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86aceaa-427e-4b66-8c55-09617823e909 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:06.549116 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86aceaa-427e-4b66-8c55-09617823e909 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:06.549421 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86aceaa-427e-4b66-8c55-09617823e909 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:06.549690 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86aceaa-427e-4b66-8c55-09617823e909 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:06.551307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f86aceaa-427e-4b66-8c55-09617823e909 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:06.660853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86aceaa-427e-4b66-8c55-09617823e909 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:06.662552 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86aceaa-427e-4b66-8c55-09617823e909 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:06.696175 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 675/1403] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:08.817256 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5090cce-74ea-4a37-9681-77dbf63310f5 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:08.817826 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5090cce-74ea-4a37-9681-77dbf63310f5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:08.817826 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5090cce-74ea-4a37-9681-77dbf63310f5 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:08.819426 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 729/1404] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:08.824557 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720e102-30b0-490a-a45f-875cbb72c00c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:08.825000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720e102-30b0-490a-a45f-875cbb72c00c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:08.825188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720e102-30b0-490a-a45f-875cbb72c00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:08.898068 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-f720e102-30b0-490a-a45f-875cbb72c00c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:08.899672 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f720e102-30b0-490a-a45f-875cbb72c00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:08.918565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:09.019974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 676/1405] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:09.309922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5090cce-74ea-4a37-9681-77dbf63310f5 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:09.311682 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c5090cce-74ea-4a37-9681-77dbf63310f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:09.420758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:09.485076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5090cce-74ea-4a37-9681-77dbf63310f5 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:09.629281 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387031a-e65b-4a78-acce-eb707cae970e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:09.629809 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387031a-e65b-4a78-acce-eb707cae970e service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:09.630096 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387031a-e65b-4a78-acce-eb707cae970e service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:09.630375 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387031a-e65b-4a78-acce-eb707cae970e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:09.632051 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9387031a-e65b-4a78-acce-eb707cae970e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:09.707893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:09.740447 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387031a-e65b-4a78-acce-eb707cae970e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:09.742397 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387031a-e65b-4a78-acce-eb707cae970e service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:09.764102 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 730/1406] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:11.863199 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca3d97-3226-47f4-a5eb-f8804a003444 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:11.863781 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca3d97-3226-47f4-a5eb-f8804a003444 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:11.863781 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca3d97-3226-47f4-a5eb-f8804a003444 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:11.869853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 677/1407] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:26:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:11.871261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662023f9-10a3-458c-84b1-7f14113d13d8 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:11.871707 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662023f9-10a3-458c-84b1-7f14113d13d8 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:11.871859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662023f9-10a3-458c-84b1-7f14113d13d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:11.977633 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-662023f9-10a3-458c-84b1-7f14113d13d8 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:11.979331 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-662023f9-10a3-458c-84b1-7f14113d13d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:12.107970 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 731/1408] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:12.406186 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca3d97-3226-47f4-a5eb-f8804a003444 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:12.407843 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f2ca3d97-3226-47f4-a5eb-f8804a003444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:12.579536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:12.581403 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca3d97-3226-47f4-a5eb-f8804a003444 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:12.723528 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8dd045-5061-4129-b9fb-e446952146c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:12.724170 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8dd045-5061-4129-b9fb-e446952146c6 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:12.724456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8dd045-5061-4129-b9fb-e446952146c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:12.724725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8dd045-5061-4129-b9fb-e446952146c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:12.726439 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bb8dd045-5061-4129-b9fb-e446952146c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:12.832186 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8dd045-5061-4129-b9fb-e446952146c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:12.833756 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8dd045-5061-4129-b9fb-e446952146c6 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:12.855046 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 678/1409] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:22.106151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=82 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:22.852117 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:27.832841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b83efa-cc70-4b2e-be0a-85fd594e2886 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:27.833382 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b83efa-cc70-4b2e-be0a-85fd594e2886 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:27.833508 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b83efa-cc70-4b2e-be0a-85fd594e2886 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:27.834778 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 732/1410] 10.222.0.18 () {54 vars in 918 bytes} [Mon Aug 29 20:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:27.839983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbd836-bf28-43c2-845b-a2ff48e7eecf None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:27.840415 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbd836-bf28-43c2-845b-a2ff48e7eecf None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:27.840546 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbd836-bf28-43c2-845b-a2ff48e7eecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:27.916565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5acbd836-bf28-43c2-845b-a2ff48e7eecf None None] MFA Rules not processed for user `fa0e659eee5e4a9da8d3d7337cb73798`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:27.917782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5acbd836-bf28-43c2-845b-a2ff48e7eecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:28.022527 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 679/1411] 10.222.0.18 () {58 vars in 1015 bytes} [Mon Aug 29 20:26:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:28.037623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6695fea-b063-46c2-a151-f75ea56055c5 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:28.037914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6695fea-b063-46c2-a151-f75ea56055c5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:28.038207 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6695fea-b063-46c2-a151-f75ea56055c5 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:28.039575 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 733/1412] 10.222.0.18 () {54 vars in 918 bytes} [Mon Aug 29 20:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:28.044167 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d44337-cce7-466f-a615-ca0579028f2a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:28.044451 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d44337-cce7-466f-a615-ca0579028f2a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:28.044722 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d44337-cce7-466f-a615-ca0579028f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:28.115369 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:28.118545 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-77d44337-cce7-466f-a615-ca0579028f2a None None] MFA Rules not processed for user `c0c44754a1c04a2b87081aba6cb509d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:28.120373 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-77d44337-cce7-466f-a615-ca0579028f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:28.234622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 680/1413] 10.222.0.18 () {58 vars in 1015 bytes} [Mon Aug 29 20:26:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:30.672860 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703193e-bb66-4f48-93a7-8c3f10722497 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:30.673376 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703193e-bb66-4f48-93a7-8c3f10722497 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:30.673376 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703193e-bb66-4f48-93a7-8c3f10722497 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:30.674637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 734/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:30.680028 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe3290-ba9a-498b-b958-0ed26b1f21f0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:30.680441 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe3290-ba9a-498b-b958-0ed26b1f21f0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:30.680600 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe3290-ba9a-498b-b958-0ed26b1f21f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:30.747273 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-83fe3290-ba9a-498b-b958-0ed26b1f21f0 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:30.748640 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-83fe3290-ba9a-498b-b958-0ed26b1f21f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:30.805228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:30.847028 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 681/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:31.132414 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703193e-bb66-4f48-93a7-8c3f10722497 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:31.134028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3703193e-bb66-4f48-93a7-8c3f10722497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:31.280483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703193e-bb66-4f48-93a7-8c3f10722497 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:31.354079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:31.394584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458b4d9-69cb-4963-b3e2-0a3a9da45c0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:31.395115 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458b4d9-69cb-4963-b3e2-0a3a9da45c0a service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:31.395401 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458b4d9-69cb-4963-b3e2-0a3a9da45c0a service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:31.395674 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458b4d9-69cb-4963-b3e2-0a3a9da45c0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:31.397289 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d458b4d9-69cb-4963-b3e2-0a3a9da45c0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:31.485063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d458b4d9-69cb-4963-b3e2-0a3a9da45c0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:31.486651 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d458b4d9-69cb-4963-b3e2-0a3a9da45c0a service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:31.504240 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 735/1416] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:32.403500 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32272d-e0be-4d8e-ac64-740cf1cbaa34 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:32.404084 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32272d-e0be-4d8e-ac64-740cf1cbaa34 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:32.404084 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32272d-e0be-4d8e-ac64-740cf1cbaa34 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:32.405978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 682/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:26:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:32.411062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56410e9c-c2a3-4e7b-a5bb-c6c93bd4fda9 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:32.411461 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56410e9c-c2a3-4e7b-a5bb-c6c93bd4fda9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:32.411649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56410e9c-c2a3-4e7b-a5bb-c6c93bd4fda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:32.488583 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-56410e9c-c2a3-4e7b-a5bb-c6c93bd4fda9 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:32.489807 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-56410e9c-c2a3-4e7b-a5bb-c6c93bd4fda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:32.516631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:32.585905 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 736/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:32.755683 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32272d-e0be-4d8e-ac64-740cf1cbaa34 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:32.757223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-3b32272d-e0be-4d8e-ac64-740cf1cbaa34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:32.900364 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32272d-e0be-4d8e-ac64-740cf1cbaa34 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:32.940321 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:33.012466 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76d3c1-763e-46a8-95ee-25ca1bcc71ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:33.013067 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76d3c1-763e-46a8-95ee-25ca1bcc71ef service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:33.013342 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76d3c1-763e-46a8-95ee-25ca1bcc71ef service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:33.013607 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76d3c1-763e-46a8-95ee-25ca1bcc71ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:33.015262 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5b76d3c1-763e-46a8-95ee-25ca1bcc71ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:33.100992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76d3c1-763e-46a8-95ee-25ca1bcc71ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:33.102572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76d3c1-763e-46a8-95ee-25ca1bcc71ef service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:33.119883 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 683/1419] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:34.003496 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cfb97-588b-4e8e-9739-d8ce831caee3 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:34.004114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cfb97-588b-4e8e-9739-d8ce831caee3 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:34.004114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cfb97-588b-4e8e-9739-d8ce831caee3 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:34.005499 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 737/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:34.010461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc096531-cb9f-4f01-9172-085ea9515ad2 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:34.010893 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc096531-cb9f-4f01-9172-085ea9515ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:34.010998 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc096531-cb9f-4f01-9172-085ea9515ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:34.066546 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:34.089058 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-cc096531-cb9f-4f01-9172-085ea9515ad2 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:34.090339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-cc096531-cb9f-4f01-9172-085ea9515ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:34.185140 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 684/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:26:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:34.348684 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470cfb97-588b-4e8e-9739-d8ce831caee3 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:34.350220 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-470cfb97-588b-4e8e-9739-d8ce831caee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:34.494906 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470cfb97-588b-4e8e-9739-d8ce831caee3 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:34.507635 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:34.608158 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6538fbb5-75f8-42cb-a879-70b30eb702a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:34.608688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538fbb5-75f8-42cb-a879-70b30eb702a6 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:34.608970 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538fbb5-75f8-42cb-a879-70b30eb702a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:34.609243 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6538fbb5-75f8-42cb-a879-70b30eb702a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:34.610839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6538fbb5-75f8-42cb-a879-70b30eb702a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:34.701824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6538fbb5-75f8-42cb-a879-70b30eb702a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:34.703391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6538fbb5-75f8-42cb-a879-70b30eb702a6 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:34.721273 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 738/1422] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:35.593290 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8d9e8-2528-4274-813d-adedc4932f2d None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:35.593828 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8d9e8-2528-4274-813d-adedc4932f2d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:35.593828 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8d9e8-2528-4274-813d-adedc4932f2d None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:35.595171 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 685/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:26:35.600225 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0b248-33e0-492d-b1fc-34ad0c9c2a18 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:35.600614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0b248-33e0-492d-b1fc-34ad0c9c2a18 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:35.600814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0b248-33e0-492d-b1fc-34ad0c9c2a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:35.621101 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:35.678450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-38a0b248-33e0-492d-b1fc-34ad0c9c2a18 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:35.679643 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-38a0b248-33e0-492d-b1fc-34ad0c9c2a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:35.777002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 739/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:35.929214 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8d9e8-2528-4274-813d-adedc4932f2d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:35.930640 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-92d8d9e8-2528-4274-813d-adedc4932f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:36.038616 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:36.074725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8d9e8-2528-4274-813d-adedc4932f2d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:36.185294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cb351-3fbe-4122-be1a-9207f3708924 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:36.186414 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cb351-3fbe-4122-be1a-9207f3708924 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:36.186691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cb351-3fbe-4122-be1a-9207f3708924 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:36.186960 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cb351-3fbe-4122-be1a-9207f3708924 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:36.188561 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f83cb351-3fbe-4122-be1a-9207f3708924 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:36.263312 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:36.276963 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83cb351-3fbe-4122-be1a-9207f3708924 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:36.278657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83cb351-3fbe-4122-be1a-9207f3708924 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:36.297632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 686/1425] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:37.168056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3931934-efe1-4af1-97cb-4206c5c508d7 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:37.168576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3931934-efe1-4af1-97cb-4206c5c508d7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:37.168713 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3931934-efe1-4af1-97cb-4206c5c508d7 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:37.170789 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 740/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:26:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:37.175720 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40091c-7d32-4bf2-832e-99c8728dc88a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:37.176172 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40091c-7d32-4bf2-832e-99c8728dc88a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:37.176317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40091c-7d32-4bf2-832e-99c8728dc88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:37.254263 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1b40091c-7d32-4bf2-832e-99c8728dc88a None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:37.255493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1b40091c-7d32-4bf2-832e-99c8728dc88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:37.354801 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 687/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:37.504249 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3931934-efe1-4af1-97cb-4206c5c508d7 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:37.505707 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b3931934-efe1-4af1-97cb-4206c5c508d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:37.591510 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:37.656101 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3931934-efe1-4af1-97cb-4206c5c508d7 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:37.767855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106da681-4ae0-412d-8590-de3d3f643450 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:37.768430 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106da681-4ae0-412d-8590-de3d3f643450 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:37.768743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106da681-4ae0-412d-8590-de3d3f643450 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:37.769022 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106da681-4ae0-412d-8590-de3d3f643450 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:37.770665 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-106da681-4ae0-412d-8590-de3d3f643450 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:37.819487 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:37.860392 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106da681-4ae0-412d-8590-de3d3f643450 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:37.861981 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106da681-4ae0-412d-8590-de3d3f643450 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:37.879380 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 741/1428] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:38.706968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e27981-f691-4b6c-9c8f-13fa3e1b23bf None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:38.707498 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e27981-f691-4b6c-9c8f-13fa3e1b23bf None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:38.707498 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e27981-f691-4b6c-9c8f-13fa3e1b23bf None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:38.708839 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 688/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:38.713711 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc3290-e11b-4fd6-934c-7b97fcaa2855 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:38.714120 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc3290-e11b-4fd6-934c-7b97fcaa2855 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:38.714332 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc3290-e11b-4fd6-934c-7b97fcaa2855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:38.791502 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-93bc3290-e11b-4fd6-934c-7b97fcaa2855 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:38.792739 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-93bc3290-e11b-4fd6-934c-7b97fcaa2855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:38.890374 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 742/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:39.040120 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e27981-f691-4b6c-9c8f-13fa3e1b23bf None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:39.041624 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e7e27981-f691-4b6c-9c8f-13fa3e1b23bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:39.089904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:39.185053 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e27981-f691-4b6c-9c8f-13fa3e1b23bf None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:39.295911 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c6a03-9ff9-4e9e-8c73-e5f9474e7871 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:39.296462 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c6a03-9ff9-4e9e-8c73-e5f9474e7871 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:39.296739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c6a03-9ff9-4e9e-8c73-e5f9474e7871 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:39.297013 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c6a03-9ff9-4e9e-8c73-e5f9474e7871 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:39.298591 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c72c6a03-9ff9-4e9e-8c73-e5f9474e7871 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:39.306134 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:39.390934 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c6a03-9ff9-4e9e-8c73-e5f9474e7871 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:39.392578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c6a03-9ff9-4e9e-8c73-e5f9474e7871 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:39.410186 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 689/1431] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:40.234740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df1923-12ed-4aad-bcfc-a61a736e9f7b None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:40.235259 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df1923-12ed-4aad-bcfc-a61a736e9f7b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:40.235259 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df1923-12ed-4aad-bcfc-a61a736e9f7b None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:40.236604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 743/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:26:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:40.241594 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca2de1-ad61-45c8-b7f1-079e1b3fe94f None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:40.242005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca2de1-ad61-45c8-b7f1-079e1b3fe94f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:40.242187 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca2de1-ad61-45c8-b7f1-079e1b3fe94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:40.321155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-01ca2de1-ad61-45c8-b7f1-079e1b3fe94f None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:40.322371 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-01ca2de1-ad61-45c8-b7f1-079e1b3fe94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:40.390203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:40.419057 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 690/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:40.573075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df1923-12ed-4aad-bcfc-a61a736e9f7b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:40.574725 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-41df1923-12ed-4aad-bcfc-a61a736e9f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:40.592926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:40.724606 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df1923-12ed-4aad-bcfc-a61a736e9f7b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:40.813275 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:40.844805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba86b9d-d62b-4621-8d57-6d28a4a1eb1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:40.845338 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba86b9d-d62b-4621-8d57-6d28a4a1eb1a service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:40.845620 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba86b9d-d62b-4621-8d57-6d28a4a1eb1a service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:40.845897 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba86b9d-d62b-4621-8d57-6d28a4a1eb1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:40.847498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-8ba86b9d-d62b-4621-8d57-6d28a4a1eb1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:40.938877 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba86b9d-d62b-4621-8d57-6d28a4a1eb1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:40.940512 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba86b9d-d62b-4621-8d57-6d28a4a1eb1a service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:40.957974 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 744/1434] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:41.775797 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d1482-25cb-411e-9c9b-e3d53df8d719 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:41.776362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d1482-25cb-411e-9c9b-e3d53df8d719 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:41.776483 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d1482-25cb-411e-9c9b-e3d53df8d719 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:41.777778 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 691/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:41.783227 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e9268-3600-4d64-a169-28cc73636764 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:41.783665 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e9268-3600-4d64-a169-28cc73636764 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:41.783764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e9268-3600-4d64-a169-28cc73636764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:41.859846 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-920e9268-3600-4d64-a169-28cc73636764 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:41.861139 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-920e9268-3600-4d64-a169-28cc73636764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:41.902285 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:41.958538 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 745/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:42.112463 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d1482-25cb-411e-9c9b-e3d53df8d719 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:42.114001 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7d2d1482-25cb-411e-9c9b-e3d53df8d719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:42.261036 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d1482-25cb-411e-9c9b-e3d53df8d719 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:42.318771 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:42.377886 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d27a4b-d46f-4461-9068-e9741fd1bf6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:42.378412 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d27a4b-d46f-4461-9068-e9741fd1bf6a service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:42.378685 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d27a4b-d46f-4461-9068-e9741fd1bf6a service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:42.378955 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d27a4b-d46f-4461-9068-e9741fd1bf6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:42.380545 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-23d27a4b-d46f-4461-9068-e9741fd1bf6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:42.470728 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d27a4b-d46f-4461-9068-e9741fd1bf6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:42.472354 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d27a4b-d46f-4461-9068-e9741fd1bf6a service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:42.489771 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 692/1437] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:43.289831 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813f61e-bb64-41cf-8fba-281a06c77542 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:43.290368 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813f61e-bb64-41cf-8fba-281a06c77542 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:43.290368 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813f61e-bb64-41cf-8fba-281a06c77542 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:43.291712 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 746/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:43.296767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077e9c4-4469-4ed5-b905-833310a328dc None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:43.297211 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077e9c4-4469-4ed5-b905-833310a328dc None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:43.297322 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077e9c4-4469-4ed5-b905-833310a328dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:43.376791 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c077e9c4-4469-4ed5-b905-833310a328dc None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:43.377999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c077e9c4-4469-4ed5-b905-833310a328dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:43.380079 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:43.473964 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 693/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:26:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:26:43.625224 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813f61e-bb64-41cf-8fba-281a06c77542 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:43.626760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b813f61e-bb64-41cf-8fba-281a06c77542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:43.771986 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813f61e-bb64-41cf-8fba-281a06c77542 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:43.798834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:43.884117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e7ce8-a9d7-463d-a05a-c6019c6649cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:43.884637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e7ce8-a9d7-463d-a05a-c6019c6649cb service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:43.884920 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e7ce8-a9d7-463d-a05a-c6019c6649cb service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:43.885196 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e7ce8-a9d7-463d-a05a-c6019c6649cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:43.886784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-de3e7ce8-a9d7-463d-a05a-c6019c6649cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:43.975382 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e7ce8-a9d7-463d-a05a-c6019c6649cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:43.977041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e7ce8-a9d7-463d-a05a-c6019c6649cb service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:43.994663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 747/1440] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:45.828307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fa63a-e294-4ebb-bed9-70057c464e36 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:45.828841 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fa63a-e294-4ebb-bed9-70057c464e36 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:45.828841 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fa63a-e294-4ebb-bed9-70057c464e36 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:45.830203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 694/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:45.835503 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a0a29-2930-4b8b-a247-19bb4aefd0bc None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:45.835949 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a0a29-2930-4b8b-a247-19bb4aefd0bc None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:45.836088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a0a29-2930-4b8b-a247-19bb4aefd0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:45.878343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:45.904059 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-101a0a29-2930-4b8b-a247-19bb4aefd0bc None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:45.905266 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-101a0a29-2930-4b8b-a247-19bb4aefd0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:46.005688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 748/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:46.287147 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fa63a-e294-4ebb-bed9-70057c464e36 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:46.288816 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-975fa63a-e294-4ebb-bed9-70057c464e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:46.392252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fa63a-e294-4ebb-bed9-70057c464e36 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:46.424599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:46.465045 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01815116-c3a9-45cc-bd0d-690c0e0f1539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:46.465709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01815116-c3a9-45cc-bd0d-690c0e0f1539 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:46.465985 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01815116-c3a9-45cc-bd0d-690c0e0f1539 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:46.466252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01815116-c3a9-45cc-bd0d-690c0e0f1539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:46.468493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-01815116-c3a9-45cc-bd0d-690c0e0f1539 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:46.576558 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01815116-c3a9-45cc-bd0d-690c0e0f1539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:46.577955 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01815116-c3a9-45cc-bd0d-690c0e0f1539 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:46.596777 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 695/1443] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:48.665941 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86943b98-e0b2-4fc7-bd9c-1ccaaf50a379 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:48.666483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86943b98-e0b2-4fc7-bd9c-1ccaaf50a379 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:48.666483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86943b98-e0b2-4fc7-bd9c-1ccaaf50a379 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:48.667761 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 749/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:48.673002 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a212246-1d02-4db5-8691-484c3f80f41a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:48.673412 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a212246-1d02-4db5-8691-484c3f80f41a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:48.673570 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a212246-1d02-4db5-8691-484c3f80f41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:48.739813 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5a212246-1d02-4db5-8691-484c3f80f41a None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:48.741053 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5a212246-1d02-4db5-8691-484c3f80f41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:48.754428 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:48.838785 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 696/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:49.119734 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86943b98-e0b2-4fc7-bd9c-1ccaaf50a379 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:49.121339 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-86943b98-e0b2-4fc7-bd9c-1ccaaf50a379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:49.223797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86943b98-e0b2-4fc7-bd9c-1ccaaf50a379 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:49.236334 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:49.298900 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53809c21-ca91-4ad8-b750-b5eb125d8817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:49.299513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53809c21-ca91-4ad8-b750-b5eb125d8817 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:49.299796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53809c21-ca91-4ad8-b750-b5eb125d8817 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:49.300105 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53809c21-ca91-4ad8-b750-b5eb125d8817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:49.301714 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-53809c21-ca91-4ad8-b750-b5eb125d8817 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:49.411474 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53809c21-ca91-4ad8-b750-b5eb125d8817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:49.413134 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53809c21-ca91-4ad8-b750-b5eb125d8817 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:49.430885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 750/1446] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:51.678602 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9049a-bbe7-4aa4-9b84-3dc22958099f None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:51.679143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9049a-bbe7-4aa4-9b84-3dc22958099f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:51.679143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9049a-bbe7-4aa4-9b84-3dc22958099f None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:51.680436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 697/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:51.685697 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776b922-1d1c-42e3-b40d-07e5c8ca96ab None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:51.686103 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776b922-1d1c-42e3-b40d-07e5c8ca96ab None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:51.686267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776b922-1d1c-42e3-b40d-07e5c8ca96ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:51.736657 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:51.755119 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-5776b922-1d1c-42e3-b40d-07e5c8ca96ab None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:51.756405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5776b922-1d1c-42e3-b40d-07e5c8ca96ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:51.852898 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 751/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:52.136026 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9049a-bbe7-4aa4-9b84-3dc22958099f None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:52.137549 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2ed9049a-bbe7-4aa4-9b84-3dc22958099f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:52.240153 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9049a-bbe7-4aa4-9b84-3dc22958099f None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:52.280550 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:52.316651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc00440-e7fb-4ff8-9ce0-ee8f2496d0dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:52.317266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc00440-e7fb-4ff8-9ce0-ee8f2496d0dc service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:52.317544 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc00440-e7fb-4ff8-9ce0-ee8f2496d0dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:52.317817 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc00440-e7fb-4ff8-9ce0-ee8f2496d0dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:52.319390 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7bc00440-e7fb-4ff8-9ce0-ee8f2496d0dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:52.427225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc00440-e7fb-4ff8-9ce0-ee8f2496d0dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:52.428687 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc00440-e7fb-4ff8-9ce0-ee8f2496d0dc service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:52.447078 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 698/1449] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:54.539857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffed7d1-3751-4323-a991-05559ad4de08 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.540452 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffed7d1-3751-4323-a991-05559ad4de08 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.540452 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffed7d1-3751-4323-a991-05559ad4de08 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.541801 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 752/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:54.548065 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb42b8-235f-4b05-9dd4-34b1943a1d76 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:54.548484 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb42b8-235f-4b05-9dd4-34b1943a1d76 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:54.548626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb42b8-235f-4b05-9dd4-34b1943a1d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:54.613926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-05fb42b8-235f-4b05-9dd4-34b1943a1d76 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:54.615067 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-05fb42b8-235f-4b05-9dd4-34b1943a1d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:54.635166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:54.711915 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 699/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:26:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:55.000392 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffed7d1-3751-4323-a991-05559ad4de08 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:55.001921 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2ffed7d1-3751-4323-a991-05559ad4de08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:55.103470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffed7d1-3751-4323-a991-05559ad4de08 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:55.124200 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:55.177892 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fd1bb-430b-4bc2-88d3-598648238c78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:55.178499 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fd1bb-430b-4bc2-88d3-598648238c78 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:55.178773 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fd1bb-430b-4bc2-88d3-598648238c78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:55.179043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fd1bb-430b-4bc2-88d3-598648238c78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:55.180636 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-177fd1bb-430b-4bc2-88d3-598648238c78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:55.288313 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fd1bb-430b-4bc2-88d3-598648238c78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:55.289758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fd1bb-430b-4bc2-88d3-598648238c78 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:55.308503 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 753/1452] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:26:57.260542 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25713503-1d2a-4174-b6f6-6aaf767b54b7 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:57.261088 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25713503-1d2a-4174-b6f6-6aaf767b54b7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:57.261209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25713503-1d2a-4174-b6f6-6aaf767b54b7 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:57.262629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 700/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:26:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:26:57.267973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa8359-9a69-4c2f-8998-2f4053d6ed1c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:57.268399 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa8359-9a69-4c2f-8998-2f4053d6ed1c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:57.268553 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa8359-9a69-4c2f-8998-2f4053d6ed1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:57.334142 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:57.337002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-20fa8359-9a69-4c2f-8998-2f4053d6ed1c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:26:57.338777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-20fa8359-9a69-4c2f-8998-2f4053d6ed1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:57.440277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 754/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:26:57.725060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25713503-1d2a-4174-b6f6-6aaf767b54b7 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:26:57.726594 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-25713503-1d2a-4174-b6f6-6aaf767b54b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:57.829814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25713503-1d2a-4174-b6f6-6aaf767b54b7 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:26:57.878564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:26:57.904391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1146998-33ff-4efc-a2a4-ffe3c601b26e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:26:57.904993 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1146998-33ff-4efc-a2a4-ffe3c601b26e service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:26:57.905278 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1146998-33ff-4efc-a2a4-ffe3c601b26e service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:26:57.905546 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1146998-33ff-4efc-a2a4-ffe3c601b26e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:26:57.907156 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a1146998-33ff-4efc-a2a4-ffe3c601b26e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:26:58.017344 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1146998-33ff-4efc-a2a4-ffe3c601b26e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:26:58.018794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1146998-33ff-4efc-a2a4-ffe3c601b26e service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:26:58.037760 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 701/1455] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:26:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:27:00.040017 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73941352-60f0-42b4-a4bc-70b23c2204d8 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:00.040582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73941352-60f0-42b4-a4bc-70b23c2204d8 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:00.040582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73941352-60f0-42b4-a4bc-70b23c2204d8 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:00.041924 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 755/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:27:00.047514 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690bf83-99d1-4767-8af2-465543eef027 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:00.048015 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690bf83-99d1-4767-8af2-465543eef027 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:00.048110 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690bf83-99d1-4767-8af2-465543eef027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:00.116496 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-4690bf83-99d1-4767-8af2-465543eef027 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:00.118691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4690bf83-99d1-4767-8af2-465543eef027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:00.149769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:00.219034 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 702/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:00.502740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73941352-60f0-42b4-a4bc-70b23c2204d8 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:00.504347 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-73941352-60f0-42b4-a4bc-70b23c2204d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:00.620500 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73941352-60f0-42b4-a4bc-70b23c2204d8 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:00.651072 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:00.698779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0032cc6-53ac-4806-95bf-2fbd6ab3e76a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:00.699388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0032cc6-53ac-4806-95bf-2fbd6ab3e76a service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:00.699693 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0032cc6-53ac-4806-95bf-2fbd6ab3e76a service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:00.700574 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0032cc6-53ac-4806-95bf-2fbd6ab3e76a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:00.702188 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b0032cc6-53ac-4806-95bf-2fbd6ab3e76a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:00.831964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0032cc6-53ac-4806-95bf-2fbd6ab3e76a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:00.833425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0032cc6-53ac-4806-95bf-2fbd6ab3e76a service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:00.851601 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 756/1458] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:02.857539 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0357a-1a63-4703-9119-7ae19c52d303 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:02.858107 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0357a-1a63-4703-9119-7ae19c52d303 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:02.858107 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0357a-1a63-4703-9119-7ae19c52d303 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:02.859443 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 703/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:27:02.864646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87489042-bcd2-4265-99a8-af9da7d749cd None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:02.865092 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87489042-bcd2-4265-99a8-af9da7d749cd None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:02.865221 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87489042-bcd2-4265-99a8-af9da7d749cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:02.931334 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-87489042-bcd2-4265-99a8-af9da7d749cd None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:02.932744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-87489042-bcd2-4265-99a8-af9da7d749cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:02.942751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:03.033231 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 757/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:03.322094 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0357a-1a63-4703-9119-7ae19c52d303 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:03.323624 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-25d0357a-1a63-4703-9119-7ae19c52d303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:03.427781 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0357a-1a63-4703-9119-7ae19c52d303 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:03.484958 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:03.503687 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c62def-4461-409f-babd-853f1fa81cff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:03.504337 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c62def-4461-409f-babd-853f1fa81cff service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:03.504623 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c62def-4461-409f-babd-853f1fa81cff service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:03.504899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c62def-4461-409f-babd-853f1fa81cff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:03.506512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e5c62def-4461-409f-babd-853f1fa81cff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:03.615672 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c62def-4461-409f-babd-853f1fa81cff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:03.617047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c62def-4461-409f-babd-853f1fa81cff service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:03.635193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 704/1461] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:05.629561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a40b64-994e-4885-a2ba-c524b7d6c606 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:05.630123 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a40b64-994e-4885-a2ba-c524b7d6c606 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:05.630123 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a40b64-994e-4885-a2ba-c524b7d6c606 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:05.631353 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 758/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:27:05.636536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02adc2-83eb-4ba8-bf6a-a1403d931d01 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:05.637515 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02adc2-83eb-4ba8-bf6a-a1403d931d01 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:05.637618 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02adc2-83eb-4ba8-bf6a-a1403d931d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:05.702999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6b02adc2-83eb-4ba8-bf6a-a1403d931d01 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:05.704265 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6b02adc2-83eb-4ba8-bf6a-a1403d931d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:05.743851 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:05.803068 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 705/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:06.085950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a40b64-994e-4885-a2ba-c524b7d6c606 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:06.087460 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-50a40b64-994e-4885-a2ba-c524b7d6c606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:06.189751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a40b64-994e-4885-a2ba-c524b7d6c606 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:06.227551 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:06.263043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb315303-2d99-4688-8c46-25a6286e8d59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:06.263646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb315303-2d99-4688-8c46-25a6286e8d59 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:06.263957 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb315303-2d99-4688-8c46-25a6286e8d59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:06.264233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb315303-2d99-4688-8c46-25a6286e8d59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:06.265818 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-cb315303-2d99-4688-8c46-25a6286e8d59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:06.376651 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb315303-2d99-4688-8c46-25a6286e8d59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:06.378086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb315303-2d99-4688-8c46-25a6286e8d59 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:06.395604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 759/1464] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:08.359759 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fef53f-c6f5-41dc-8524-48985131e7a0 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:08.360336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fef53f-c6f5-41dc-8524-48985131e7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:08.360336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fef53f-c6f5-41dc-8524-48985131e7a0 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:08.361931 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 706/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:27:08.367156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ff6f3-12df-4fb8-9020-660d536664e0 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:08.367578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ff6f3-12df-4fb8-9020-660d536664e0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:08.367743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ff6f3-12df-4fb8-9020-660d536664e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:08.435460 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-8d0ff6f3-12df-4fb8-9020-660d536664e0 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:08.436724 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-8d0ff6f3-12df-4fb8-9020-660d536664e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:08.454949 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:08.533909 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 760/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:08.814155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fef53f-c6f5-41dc-8524-48985131e7a0 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:08.815720 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-16fef53f-c6f5-41dc-8524-48985131e7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:08.919443 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fef53f-c6f5-41dc-8524-48985131e7a0 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:08.986678 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:08.993650 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84094bc-66e8-4c2e-ae19-bef2fb81d0f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:08.994256 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84094bc-66e8-4c2e-ae19-bef2fb81d0f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:08.994533 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84094bc-66e8-4c2e-ae19-bef2fb81d0f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:08.994805 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84094bc-66e8-4c2e-ae19-bef2fb81d0f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:08.996929 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d84094bc-66e8-4c2e-ae19-bef2fb81d0f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:09.105903 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84094bc-66e8-4c2e-ae19-bef2fb81d0f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:09.107265 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84094bc-66e8-4c2e-ae19-bef2fb81d0f9 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:09.125428 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 707/1467] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:11.045466 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d1aeb-e6c7-4d71-ae15-2a356acc090f None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:11.046015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d1aeb-e6c7-4d71-ae15-2a356acc090f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:11.046015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d1aeb-e6c7-4d71-ae15-2a356acc090f None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:11.047368 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 761/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:27:11.052831 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a10bc1-0175-48cb-9280-b07ae7b2a2b4 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:11.053262 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a10bc1-0175-48cb-9280-b07ae7b2a2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:11.053364 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a10bc1-0175-48cb-9280-b07ae7b2a2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:11.120172 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-39a10bc1-0175-48cb-9280-b07ae7b2a2b4 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:11.122057 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-39a10bc1-0175-48cb-9280-b07ae7b2a2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:11.175683 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:11.228269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 708/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:11.527496 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d1aeb-e6c7-4d71-ae15-2a356acc090f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:11.529058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-835d1aeb-e6c7-4d71-ae15-2a356acc090f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:11.632700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d1aeb-e6c7-4d71-ae15-2a356acc090f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:11.679204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:11.705949 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bed640-1a87-4947-9c53-eb511070b23d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:11.706560 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bed640-1a87-4947-9c53-eb511070b23d service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:11.706841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bed640-1a87-4947-9c53-eb511070b23d service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:11.707116 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bed640-1a87-4947-9c53-eb511070b23d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:11.708737 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-07bed640-1a87-4947-9c53-eb511070b23d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:11.821258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bed640-1a87-4947-9c53-eb511070b23d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:11.822731 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bed640-1a87-4947-9c53-eb511070b23d service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:11.841988 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 762/1470] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:21.226409 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=22 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:21.839219 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=66 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:31.460849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d7132-b666-4c87-a6be-be9c5a9a1dfa None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:31.461433 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d7132-b666-4c87-a6be-be9c5a9a1dfa None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:31.461433 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d7132-b666-4c87-a6be-be9c5a9a1dfa None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:31.462766 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 709/1471] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:27:31.468600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2b360-1186-4105-afd7-778ab560976a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:31.469009 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2b360-1186-4105-afd7-778ab560976a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:31.469169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2b360-1186-4105-afd7-778ab560976a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:31.535915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-20e2b360-1186-4105-afd7-778ab560976a None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:31.537148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-20e2b360-1186-4105-afd7-778ab560976a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:31.633581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 763/1472] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:31.936888 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561af165-6f36-49b3-9d32-12b18c9a5eb3 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:31.937169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561af165-6f36-49b3-9d32-12b18c9a5eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:31.937439 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561af165-6f36-49b3-9d32-12b18c9a5eb3 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:31.938795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 710/1473] 10.222.0.18 () {54 vars in 967 bytes} [Mon Aug 29 20:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:27:31.943642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c173b-e7d2-4a80-aa45-ab08b1cf25a0 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:31.943992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c173b-e7d2-4a80-aa45-ab08b1cf25a0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:31.944282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c173b-e7d2-4a80-aa45-ab08b1cf25a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:32.020695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-3d9c173b-e7d2-4a80-aa45-ab08b1cf25a0 None None] MFA Rules not processed for user `5d022ed8d2ac43ae816aa396a8e2f17a`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:32.021924 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3d9c173b-e7d2-4a80-aa45-ab08b1cf25a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:32.043282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:32.099044 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 764/1474] 10.222.0.18 () {58 vars in 1064 bytes} [Mon Aug 29 20:27:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:32.107272 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561af165-6f36-49b3-9d32-12b18c9a5eb3 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:32.108798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-561af165-6f36-49b3-9d32-12b18c9a5eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:32.208270 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561af165-6f36-49b3-9d32-12b18c9a5eb3 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:32.279391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e5f91-b7e3-4bcd-a045-dc29eba7358d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d022ed8d2ac43ae816aa396a8e2f17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:32.280033 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e5f91-b7e3-4bcd-a045-dc29eba7358d service glance] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:32.280313 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e5f91-b7e3-4bcd-a045-dc29eba7358d service glance] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:32.280580 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e5f91-b7e3-4bcd-a045-dc29eba7358d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:32.282178 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6f7e5f91-b7e3-4bcd-a045-dc29eba7358d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:32.336541 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:32.391670 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e5f91-b7e3-4bcd-a045-dc29eba7358d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:32.393169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e5f91-b7e3-4bcd-a045-dc29eba7358d service glance] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:32.411845 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 711/1475] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:34.402585 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702bd38-bc19-455b-9fbf-d98facf56323 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:34.403140 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702bd38-bc19-455b-9fbf-d98facf56323 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:34.403140 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4702bd38-bc19-455b-9fbf-d98facf56323 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:34.404526 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 765/1476] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:27:34.409722 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe491e-0f27-4452-a77a-7487a63ff019 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:34.410416 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe491e-0f27-4452-a77a-7487a63ff019 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:34.410569 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe491e-0f27-4452-a77a-7487a63ff019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:34.476800 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-22fe491e-0f27-4452-a77a-7487a63ff019 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:34.478015 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-22fe491e-0f27-4452-a77a-7487a63ff019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:34.573593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 712/1477] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:34.844075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702bd38-bc19-455b-9fbf-d98facf56323 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:34.845518 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4702bd38-bc19-455b-9fbf-d98facf56323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:34.968197 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4702bd38-bc19-455b-9fbf-d98facf56323 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:35.026955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:35.073080 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096b09b0-e84f-481c-8a4d-20530cbd50d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:35.073690 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b09b0-e84f-481c-8a4d-20530cbd50d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:35.073972 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b09b0-e84f-481c-8a4d-20530cbd50d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:35.074246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b09b0-e84f-481c-8a4d-20530cbd50d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:35.074882 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b09b0-e84f-481c-8a4d-20530cbd50d1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:35.076659 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: WARNING py.warnings [None req-096b09b0-e84f-481c-8a4d-20530cbd50d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:27:35.076659 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: warnings.warn(msg) Aug 29 20:27:35.076659 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: Aug 29 20:27:35.077801 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096b09b0-e84f-481c-8a4d-20530cbd50d1 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:35.096083 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 766/1478] 10.222.0.18 () {56 vars in 1108 bytes} [Mon Aug 29 20:27:34 2022] GET /identity/v3/projects => generated 2553 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:27:37.075542 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637d4b7-e09c-4e5b-a403-a3bae3cb40a0 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:37.076195 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637d4b7-e09c-4e5b-a403-a3bae3cb40a0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:37.076195 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637d4b7-e09c-4e5b-a403-a3bae3cb40a0 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:37.077551 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 713/1479] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:27:37.083293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e246f7b-9f3d-4d41-a358-2b26836e4d33 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:37.083836 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e246f7b-9f3d-4d41-a358-2b26836e4d33 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:37.083836 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e246f7b-9f3d-4d41-a358-2b26836e4d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:37.163689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-7e246f7b-9f3d-4d41-a358-2b26836e4d33 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:37.164922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7e246f7b-9f3d-4d41-a358-2b26836e4d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:37.262651 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 767/1480] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:37.557390 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637d4b7-e09c-4e5b-a403-a3bae3cb40a0 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:37.558958 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0637d4b7-e09c-4e5b-a403-a3bae3cb40a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:37.568879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:37.706884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637d4b7-e09c-4e5b-a403-a3bae3cb40a0 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:37.786060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:37.819501 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b0160-363a-4e2c-85c4-6da80e545a3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:37.820071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b0160-363a-4e2c-85c4-6da80e545a3d service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:37.820348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b0160-363a-4e2c-85c4-6da80e545a3d service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:37.820617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b0160-363a-4e2c-85c4-6da80e545a3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:37.822230 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f35b0160-363a-4e2c-85c4-6da80e545a3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:37.908341 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b0160-363a-4e2c-85c4-6da80e545a3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:37.910202 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b0160-363a-4e2c-85c4-6da80e545a3d service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:37.928736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 714/1481] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:40.018770 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf31687-63e2-4e8c-bd10-9d6bad0001b9 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:40.019327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf31687-63e2-4e8c-bd10-9d6bad0001b9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:40.019327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf31687-63e2-4e8c-bd10-9d6bad0001b9 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:40.020722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 768/1482] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:27:40.026169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e52832-4216-4d4b-aaac-556f25240b16 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:40.026612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e52832-4216-4d4b-aaac-556f25240b16 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:40.026716 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e52832-4216-4d4b-aaac-556f25240b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:40.094151 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d2e52832-4216-4d4b-aaac-556f25240b16 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:40.095412 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d2e52832-4216-4d4b-aaac-556f25240b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:40.141595 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:40.191643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 715/1483] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:40.483529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf31687-63e2-4e8c-bd10-9d6bad0001b9 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:40.485095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1bf31687-63e2-4e8c-bd10-9d6bad0001b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:40.510322 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:40.589399 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf31687-63e2-4e8c-bd10-9d6bad0001b9 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:40.665204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704dc7b-8dca-46e0-93f9-9c3013a5f1f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:40.666363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704dc7b-8dca-46e0-93f9-9c3013a5f1f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:40.666649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704dc7b-8dca-46e0-93f9-9c3013a5f1f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:40.666926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704dc7b-8dca-46e0-93f9-9c3013a5f1f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:40.668553 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1704dc7b-8dca-46e0-93f9-9c3013a5f1f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:40.743301 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:40.778557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704dc7b-8dca-46e0-93f9-9c3013a5f1f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:40.780034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704dc7b-8dca-46e0-93f9-9c3013a5f1f1 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:40.801308 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 769/1484] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:42.773731 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e7545-2bbe-472a-a888-39eefeeef8f2 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:42.774319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e7545-2bbe-472a-a888-39eefeeef8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:42.774319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e7545-2bbe-472a-a888-39eefeeef8f2 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:42.775664 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 716/1485] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:27:42.780865 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76220b7-a46b-4df9-81a9-0182cabb4647 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:42.781310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76220b7-a46b-4df9-81a9-0182cabb4647 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:42.781413 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76220b7-a46b-4df9-81a9-0182cabb4647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:42.966625 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-c76220b7-a46b-4df9-81a9-0182cabb4647 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:42.967847 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c76220b7-a46b-4df9-81a9-0182cabb4647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:43.062572 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 770/1486] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:43.362264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e7545-2bbe-472a-a888-39eefeeef8f2 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:43.363827 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ed1e7545-2bbe-472a-a888-39eefeeef8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:43.466971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e7545-2bbe-472a-a888-39eefeeef8f2 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:43.539543 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:43.541640 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a04a1b-b640-4fe9-9385-441ce8263ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:43.542432 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a04a1b-b640-4fe9-9385-441ce8263ecd service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:43.542919 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a04a1b-b640-4fe9-9385-441ce8263ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:43.543388 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a04a1b-b640-4fe9-9385-441ce8263ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:43.545130 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-18a04a1b-b640-4fe9-9385-441ce8263ecd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:43.654265 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a04a1b-b640-4fe9-9385-441ce8263ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:43.655615 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a04a1b-b640-4fe9-9385-441ce8263ecd service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:43.674386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 717/1487] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:45.747571 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c061e09-fe2e-439a-b9fa-a1530ffc0d1e None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:45.748206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c061e09-fe2e-439a-b9fa-a1530ffc0d1e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:45.748206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c061e09-fe2e-439a-b9fa-a1530ffc0d1e None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:45.749426 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 771/1488] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:27:45.754653 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd6b52-ad0b-44d1-96a2-631a55d4f9ce None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:45.755099 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd6b52-ad0b-44d1-96a2-631a55d4f9ce None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:45.755246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd6b52-ad0b-44d1-96a2-631a55d4f9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:45.821189 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-34dd6b52-ad0b-44d1-96a2-631a55d4f9ce None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:45.822413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-34dd6b52-ad0b-44d1-96a2-631a55d4f9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:45.873632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:45.919141 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 718/1489] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:27:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:46.210587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c061e09-fe2e-439a-b9fa-a1530ffc0d1e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:46.212771 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1c061e09-fe2e-439a-b9fa-a1530ffc0d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:46.247796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:46.315581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c061e09-fe2e-439a-b9fa-a1530ffc0d1e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:46.389267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf64324-cfe3-4429-8c56-897395a43b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:46.389883 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf64324-cfe3-4429-8c56-897395a43b0f service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:46.390167 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf64324-cfe3-4429-8c56-897395a43b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:46.390441 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf64324-cfe3-4429-8c56-897395a43b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:46.392057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2cf64324-cfe3-4429-8c56-897395a43b0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:46.470199 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:46.501425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf64324-cfe3-4429-8c56-897395a43b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:46.502849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf64324-cfe3-4429-8c56-897395a43b0f service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:46.520669 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 772/1490] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:48.629621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c1030-0243-46a6-938a-a79b8780dbee None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:48.630181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c1030-0243-46a6-938a-a79b8780dbee None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:48.630181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c1030-0243-46a6-938a-a79b8780dbee None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:48.631445 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 719/1491] 10.222.0.18 () {54 vars in 927 bytes} [Mon Aug 29 20:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:27:48.636826 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ff62d-c196-4813-8208-0eee05614536 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:48.637252 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ff62d-c196-4813-8208-0eee05614536 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:48.637366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ff62d-c196-4813-8208-0eee05614536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:48.703116 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-964ff62d-c196-4813-8208-0eee05614536 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:27:48.704369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-964ff62d-c196-4813-8208-0eee05614536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:48.806062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 773/1492] 10.222.0.18 () {58 vars in 1024 bytes} [Mon Aug 29 20:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:27:49.128168 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c1030-0243-46a6-938a-a79b8780dbee None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:27:49.129712 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1b2c1030-0243-46a6-938a-a79b8780dbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:49.355273 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c1030-0243-46a6-938a-a79b8780dbee None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:27:49.451047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d83b84-1993-40e2-a027-a882386668de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:27:49.451571 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d83b84-1993-40e2-a027-a882386668de service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:27:49.451847 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d83b84-1993-40e2-a027-a882386668de service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:27:49.452134 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d83b84-1993-40e2-a027-a882386668de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:27:49.453973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-59d83b84-1993-40e2-a027-a882386668de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:27:49.462781 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:49.563619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d83b84-1993-40e2-a027-a882386668de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:27:49.565087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d83b84-1993-40e2-a027-a882386668de service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:27:49.583280 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 720/1493] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:27:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:27:58.804217 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=82 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:27:59.581227 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=50 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:07.188348 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6b187-2b57-4599-9042-e301d8cee538 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:07.189099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6b187-2b57-4599-9042-e301d8cee538 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:07.189226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6b187-2b57-4599-9042-e301d8cee538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:07.264746 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-95d6b187-2b57-4599-9042-e301d8cee538 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:07.266018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-95d6b187-2b57-4599-9042-e301d8cee538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:07.363718 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 774/1494] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:29:07.371739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275e1d8-c184-449d-8091-416f8128d216 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:07.372283 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275e1d8-c184-449d-8091-416f8128d216 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:07.372389 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275e1d8-c184-449d-8091-416f8128d216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:07.447772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-4275e1d8-c184-449d-8091-416f8128d216 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:07.449131 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4275e1d8-c184-449d-8091-416f8128d216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:07.546326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 721/1495] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:29:07.552380 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6b187-2b57-4599-9042-e301d8cee538 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:07.554001 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-95d6b187-2b57-4599-9042-e301d8cee538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:07.565575 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6b187-2b57-4599-9042-e301d8cee538 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:07.627099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:07.660706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09dd87-50d3-4cc8-9a1d-e56b454e96f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:07.661313 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09dd87-50d3-4cc8-9a1d-e56b454e96f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:07.661590 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09dd87-50d3-4cc8-9a1d-e56b454e96f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:07.661863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09dd87-50d3-4cc8-9a1d-e56b454e96f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:07.662509 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09dd87-50d3-4cc8-9a1d-e56b454e96f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:29:07.662810 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09dd87-50d3-4cc8-9a1d-e56b454e96f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:07.663808 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: WARNING py.warnings [None req-6d09dd87-50d3-4cc8-9a1d-e56b454e96f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:29:07.663808 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: warnings.warn(msg) Aug 29 20:29:07.663808 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: Aug 29 20:29:07.664724 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09dd87-50d3-4cc8-9a1d-e56b454e96f9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:07.676274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 775/1496] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:29:07.684611 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275e1d8-c184-449d-8091-416f8128d216 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:07.694296 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275e1d8-c184-449d-8091-416f8128d216 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:07.755050 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:07.789386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c2d7f-3fb5-4b31-b246-494b0440a07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:07.789982 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c2d7f-3fb5-4b31-b246-494b0440a07f admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:07.790255 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c2d7f-3fb5-4b31-b246-494b0440a07f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:07.790522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c2d7f-3fb5-4b31-b246-494b0440a07f admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:07.791136 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5c2d7f-3fb5-4b31-b246-494b0440a07f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:07.792893 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: WARNING py.warnings [None req-5c5c2d7f-3fb5-4b31-b246-494b0440a07f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:29:07.792893 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: warnings.warn(msg) Aug 29 20:29:07.792893 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: Aug 29 20:29:07.794080 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5c2d7f-3fb5-4b31-b246-494b0440a07f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:07.849033 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 722/1497] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:29:07 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:29:07.853389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09dd87-50d3-4cc8-9a1d-e56b454e96f9 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:07.854913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6d09dd87-50d3-4cc8-9a1d-e56b454e96f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:07.888561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09dd87-50d3-4cc8-9a1d-e56b454e96f9 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:07.984465 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8177e-2b43-4525-bd8d-73215afc5bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:07.985080 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8177e-2b43-4525-bd8d-73215afc5bce admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:07.985364 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8177e-2b43-4525-bd8d-73215afc5bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:07.985635 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8177e-2b43-4525-bd8d-73215afc5bce admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:07.986815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e8177e-2b43-4525-bd8d-73215afc5bce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:07.987844 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: WARNING py.warnings [None req-f5e8177e-2b43-4525-bd8d-73215afc5bce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:29:07.987844 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: warnings.warn(msg) Aug 29 20:29:07.987844 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: Aug 29 20:29:07.989061 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e8177e-2b43-4525-bd8d-73215afc5bce admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:08.032319 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 776/1498] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:29:07 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:29:08.036954 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c2d7f-3fb5-4b31-b246-494b0440a07f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:08.038565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5c5c2d7f-3fb5-4b31-b246-494b0440a07f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:08.098110 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c2d7f-3fb5-4b31-b246-494b0440a07f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:08.183877 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:08.196705 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ea76f-2edb-4525-80e8-712cb511c716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:08.197309 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ea76f-2edb-4525-80e8-712cb511c716 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:08.197581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ea76f-2edb-4525-80e8-712cb511c716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:08.197844 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ea76f-2edb-4525-80e8-712cb511c716 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:08.198471 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ea76f-2edb-4525-80e8-712cb511c716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:08.199476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: WARNING py.warnings [None req-668ea76f-2edb-4525-80e8-712cb511c716 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:29:08.199476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: warnings.warn(msg) Aug 29 20:29:08.199476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: Aug 29 20:29:08.200378 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ea76f-2edb-4525-80e8-712cb511c716 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:08.213841 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 723/1499] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:29:08.218003 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8177e-2b43-4525-bd8d-73215afc5bce admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:08.219644 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f5e8177e-2b43-4525-bd8d-73215afc5bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:08.231285 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8177e-2b43-4525-bd8d-73215afc5bce admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:08.256292 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:08.331643 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7aa435-0942-4c8e-acb2-d251ffd190ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:08.332291 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aa435-0942-4c8e-acb2-d251ffd190ae admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:08.332569 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aa435-0942-4c8e-acb2-d251ffd190ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:08.332839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7aa435-0942-4c8e-acb2-d251ffd190ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:08.333487 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7aa435-0942-4c8e-acb2-d251ffd190ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:08.334550 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: WARNING py.warnings [None req-ad7aa435-0942-4c8e-acb2-d251ffd190ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:29:08.334550 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: warnings.warn(msg) Aug 29 20:29:08.334550 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: Aug 29 20:29:08.335461 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7aa435-0942-4c8e-acb2-d251ffd190ae admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:08.348370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 777/1500] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:29:08.352543 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ea76f-2edb-4525-80e8-712cb511c716 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:08.354081 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-668ea76f-2edb-4525-80e8-712cb511c716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:08.391199 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ea76f-2edb-4525-80e8-712cb511c716 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:08.484893 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd1c0f-65b8-4ed4-b663-851e9151d4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:08.485545 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd1c0f-65b8-4ed4-b663-851e9151d4d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:08.485819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd1c0f-65b8-4ed4-b663-851e9151d4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:08.486091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd1c0f-65b8-4ed4-b663-851e9151d4d6 admin admin] PATH_INFO: `/v3/projects/0c75b9419a994e6cabb69ed7cdbc32f2/users/ee816e16aaa9475db8880f7f1edf8fa0/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:08.507392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd1c0f-65b8-4ed4-b663-851e9151d4d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c75b9419a994e6cabb69ed7cdbc32f2, user_id=ee816e16aaa9475db8880f7f1edf8fa0, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:08.508453 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: WARNING py.warnings [None req-56cd1c0f-65b8-4ed4-b663-851e9151d4d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:29:08.508453 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: warnings.warn(msg) Aug 29 20:29:08.508453 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: Aug 29 20:29:08.510391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd1c0f-65b8-4ed4-b663-851e9151d4d6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:08.535242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 724/1501] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:29:08 2022] PUT /identity/v3/projects/0c75b9419a994e6cabb69ed7cdbc32f2/users/ee816e16aaa9475db8880f7f1edf8fa0/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:08.547356 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6312a-48b4-4b03-9a7e-fa5f38579e08 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:08.547658 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6312a-48b4-4b03-9a7e-fa5f38579e08 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:08.548066 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f6312a-48b4-4b03-9a7e-fa5f38579e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:08.605167 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-62f6312a-48b4-4b03-9a7e-fa5f38579e08 None None] MFA Rules not processed for user `ee816e16aaa9475db8880f7f1edf8fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:08.606269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-62f6312a-48b4-4b03-9a7e-fa5f38579e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:08.684699 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 778/1502] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:29:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:29:08.698722 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15654bd8-2aae-40db-9a9a-6e20c5a9dab3 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:08.699022 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15654bd8-2aae-40db-9a9a-6e20c5a9dab3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:08.699311 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15654bd8-2aae-40db-9a9a-6e20c5a9dab3 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:08.700740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 725/1503] 10.222.0.18 () {54 vars in 964 bytes} [Mon Aug 29 20:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:29:08.706391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28a6c6-10fe-489c-879b-7737cf215307 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:08.706677 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28a6c6-10fe-489c-879b-7737cf215307 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:08.706954 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28a6c6-10fe-489c-879b-7737cf215307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:08.737945 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:08.784983 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-3a28a6c6-10fe-489c-879b-7737cf215307 None None] MFA Rules not processed for user `85624cea63af40fda09f57ddf2cd5c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:08.786195 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3a28a6c6-10fe-489c-879b-7737cf215307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:08.859655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 779/1504] 10.222.0.18 () {58 vars in 1061 bytes} [Mon Aug 29 20:29:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:29:08.868473 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15654bd8-2aae-40db-9a9a-6e20c5a9dab3 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:08.869976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-15654bd8-2aae-40db-9a9a-6e20c5a9dab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:08.935187 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:08.990594 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15654bd8-2aae-40db-9a9a-6e20c5a9dab3 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:09.063713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3277e-c4ef-452b-8fc8-ee2aa31ba710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:09.064390 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3277e-c4ef-452b-8fc8-ee2aa31ba710 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:09.064669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3277e-c4ef-452b-8fc8-ee2aa31ba710 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:09.064935 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3277e-c4ef-452b-8fc8-ee2aa31ba710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:09.066511 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-42e3277e-c4ef-452b-8fc8-ee2aa31ba710 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:09.162246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3277e-c4ef-452b-8fc8-ee2aa31ba710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:09.163594 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3277e-c4ef-452b-8fc8-ee2aa31ba710 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:09.167054 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:09.181929 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 726/1505] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:29:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:29:09.227649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17bb2a-ec54-4ec3-87b4-51d4a8a0c7b6 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:09.228109 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17bb2a-ec54-4ec3-87b4-51d4a8a0c7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:09.228236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17bb2a-ec54-4ec3-87b4-51d4a8a0c7b6 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:09.229491 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 780/1506] 10.222.0.18 () {52 vars in 830 bytes} [Mon Aug 29 20:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:29:09.233070 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3277e-c4ef-452b-8fc8-ee2aa31ba710 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:09.234278 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-42e3277e-c4ef-452b-8fc8-ee2aa31ba710 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:09.344762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3277e-c4ef-452b-8fc8-ee2aa31ba710 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:09.437953 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd43ff2-0139-4697-a7bd-ef26e106f057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:09.438598 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd43ff2-0139-4697-a7bd-ef26e106f057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:09.438774 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd43ff2-0139-4697-a7bd-ef26e106f057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:09.439039 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd43ff2-0139-4697-a7bd-ef26e106f057 admin admin] PATH_INFO: `/v3/users/ee816e16aaa9475db8880f7f1edf8fa0` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:09.450947 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd43ff2-0139-4697-a7bd-ef26e106f057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee816e16aaa9475db8880f7f1edf8fa0)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:09.451991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: WARNING py.warnings [None req-fbd43ff2-0139-4697-a7bd-ef26e106f057 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:29:09.451991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: warnings.warn(msg) Aug 29 20:29:09.451991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: Aug 29 20:29:09.453194 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd43ff2-0139-4697-a7bd-ef26e106f057 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:09.463072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:09.525225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fbd43ff2-0139-4697-a7bd-ef26e106f057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee816e16aaa9475db8880f7f1edf8fa0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:09.540007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fbd43ff2-0139-4697-a7bd-ef26e106f057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee816e16aaa9475db8880f7f1edf8fa0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:09.549606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fbd43ff2-0139-4697-a7bd-ef26e106f057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee816e16aaa9475db8880f7f1edf8fa0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:09.550228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fbd43ff2-0139-4697-a7bd-ef26e106f057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee816e16aaa9475db8880f7f1edf8fa0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:09.573889 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fbd43ff2-0139-4697-a7bd-ef26e106f057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee816e16aaa9475db8880f7f1edf8fa0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:09.580603 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 727/1507] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:29:09 2022] DELETE /identity/v3/users/ee816e16aaa9475db8880f7f1edf8fa0 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:09.585056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81345b-6e0d-4fd0-a83d-a89a89c946b3 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:09.585346 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81345b-6e0d-4fd0-a83d-a89a89c946b3 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:09.585653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81345b-6e0d-4fd0-a83d-a89a89c946b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:09.643916 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4b81345b-6e0d-4fd0-a83d-a89a89c946b3 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:29:09.645164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4b81345b-6e0d-4fd0-a83d-a89a89c946b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:09.718270 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:09.731255 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 781/1508] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:29:09.741125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd43ff2-0139-4697-a7bd-ef26e106f057 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:09.742342 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-fbd43ff2-0139-4697-a7bd-ef26e106f057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:09.841669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd43ff2-0139-4697-a7bd-ef26e106f057 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:09.894746 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:09.915431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b80ae7-af81-443b-be0a-c8ccd5dd168f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:09.916069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b80ae7-af81-443b-be0a-c8ccd5dd168f service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:09.916364 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b80ae7-af81-443b-be0a-c8ccd5dd168f service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:09.916626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b80ae7-af81-443b-be0a-c8ccd5dd168f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:09.918184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-62b80ae7-af81-443b-be0a-c8ccd5dd168f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:10.025681 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b80ae7-af81-443b-be0a-c8ccd5dd168f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:10.027558 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b80ae7-af81-443b-be0a-c8ccd5dd168f service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:10.050265 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 728/1509] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:29:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:29:10.409860 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81345b-6e0d-4fd0-a83d-a89a89c946b3 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:29:10.411076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4b81345b-6e0d-4fd0-a83d-a89a89c946b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:29:10.422220 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81345b-6e0d-4fd0-a83d-a89a89c946b3 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:29:10.522814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a9171-d737-47d3-b0e6-76c733ed559f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:29:10.523506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a9171-d737-47d3-b0e6-76c733ed559f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:29:10.523794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a9171-d737-47d3-b0e6-76c733ed559f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:29:10.524105 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a9171-d737-47d3-b0e6-76c733ed559f admin admin] PATH_INFO: `/v3/projects/0c75b9419a994e6cabb69ed7cdbc32f2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:29:10.534177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a9171-d737-47d3-b0e6-76c733ed559f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c75b9419a994e6cabb69ed7cdbc32f2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:29:10.535204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: WARNING py.warnings [None req-264a9171-d737-47d3-b0e6-76c733ed559f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:29:10.535204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: warnings.warn(msg) Aug 29 20:29:10.535204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: Aug 29 20:29:10.536500 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a9171-d737-47d3-b0e6-76c733ed559f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:29:10.594751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-264a9171-d737-47d3-b0e6-76c733ed559f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c75b9419a994e6cabb69ed7cdbc32f2', 'request_id': 'req-264a9171-d737-47d3-b0e6-76c733ed559f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:10.603860 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-264a9171-d737-47d3-b0e6-76c733ed559f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c75b9419a994e6cabb69ed7cdbc32f2', 'request_id': 'req-264a9171-d737-47d3-b0e6-76c733ed559f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:10.610368 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-264a9171-d737-47d3-b0e6-76c733ed559f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c75b9419a994e6cabb69ed7cdbc32f2', 'request_id': 'req-264a9171-d737-47d3-b0e6-76c733ed559f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:10.643463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-264a9171-d737-47d3-b0e6-76c733ed559f admin admin] The token cache is being invalidate because project 0c75b9419a994e6cabb69ed7cdbc32f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:29:10.643825 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-264a9171-d737-47d3-b0e6-76c733ed559f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:10.644296 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-264a9171-d737-47d3-b0e6-76c733ed559f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:29:10.645920 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 782/1510] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:29:10 2022] DELETE /identity/v3/projects/0c75b9419a994e6cabb69ed7cdbc32f2 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:29:20.047772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=62 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:29:20.640812 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:30:45.670743 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b39ea-c7d6-483c-b357-84aeaf0ab706 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:45.671307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b39ea-c7d6-483c-b357-84aeaf0ab706 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:45.671307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b39ea-c7d6-483c-b357-84aeaf0ab706 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:45.672597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 729/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:30:45.677828 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29308f-5fe4-4548-861d-82c48e740ded None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:45.678342 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29308f-5fe4-4548-861d-82c48e740ded None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:45.678342 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29308f-5fe4-4548-861d-82c48e740ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:45.779319 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-9f29308f-5fe4-4548-861d-82c48e740ded None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:30:45.780603 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9f29308f-5fe4-4548-861d-82c48e740ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:45.908734 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 783/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:30:46.063903 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b39ea-c7d6-483c-b357-84aeaf0ab706 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:46.066418 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-957b39ea-c7d6-483c-b357-84aeaf0ab706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:46.223563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b39ea-c7d6-483c-b357-84aeaf0ab706 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:46.231917 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78026e8-8d15-4437-9eac-3dc4252561c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:46.231917 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78026e8-8d15-4437-9eac-3dc4252561c3 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:46.231917 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78026e8-8d15-4437-9eac-3dc4252561c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:46.231917 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78026e8-8d15-4437-9eac-3dc4252561c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:46.239739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78026e8-8d15-4437-9eac-3dc4252561c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:46.241361 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78026e8-8d15-4437-9eac-3dc4252561c3 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:46.251861 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 730/1513] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:30:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:30:47.105181 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c2e31-417d-4c8a-a708-d4a5eb24be45 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:47.105877 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c2e31-417d-4c8a-a708-d4a5eb24be45 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:47.105877 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c2e31-417d-4c8a-a708-d4a5eb24be45 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:47.107211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 784/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:30:47.112661 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0731345-c652-4855-a859-4a5bc14c6adc None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:47.113109 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0731345-c652-4855-a859-4a5bc14c6adc None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:47.113314 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0731345-c652-4855-a859-4a5bc14c6adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:47.138806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-a0731345-c652-4855-a859-4a5bc14c6adc None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:30:47.140071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a0731345-c652-4855-a859-4a5bc14c6adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:47.148476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 731/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:30:47.304523 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c2e31-417d-4c8a-a708-d4a5eb24be45 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:47.320263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c2e31-417d-4c8a-a708-d4a5eb24be45 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:47.326175 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4119835-7292-48fb-a305-e391c73fe2a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:47.326794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4119835-7292-48fb-a305-e391c73fe2a3 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:47.327079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4119835-7292-48fb-a305-e391c73fe2a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:47.327357 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4119835-7292-48fb-a305-e391c73fe2a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:47.337709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4119835-7292-48fb-a305-e391c73fe2a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:47.339328 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4119835-7292-48fb-a305-e391c73fe2a3 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:47.349773 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 785/1516] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:30:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:30:48.657562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4089b3-7341-4ce9-927c-86c63b17946a None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:48.658077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4089b3-7341-4ce9-927c-86c63b17946a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:48.658077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4089b3-7341-4ce9-927c-86c63b17946a None None] PATH_INFO: `/v3` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:48.659381 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 732/1517] 10.222.0.33 () {54 vars in 925 bytes} [Mon Aug 29 20:30:48 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:30:48.666077 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3778a-1773-451a-bde9-9ffc1cda9abc None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:48.666572 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3778a-1773-451a-bde9-9ffc1cda9abc None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:48.666677 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3778a-1773-451a-bde9-9ffc1cda9abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:48.729389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-88a3778a-1773-451a-bde9-9ffc1cda9abc None None] MFA Rules not processed for user `fa0e659eee5e4a9da8d3d7337cb73798`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:30:48.730689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-88a3778a-1773-451a-bde9-9ffc1cda9abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:48.755735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:30:48.819465 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 786/1518] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Aug 29 20:30:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:30:52.928603 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26009aa-c86e-4ad9-81a7-91ad4d58a4d9 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:52.929120 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26009aa-c86e-4ad9-81a7-91ad4d58a4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:52.929120 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26009aa-c86e-4ad9-81a7-91ad4d58a4d9 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:52.930485 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 733/1519] 10.222.0.18 () {54 vars in 961 bytes} [Mon Aug 29 20:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:30:52.935448 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f896bbf-08d1-4dc6-966a-904ec5a824fd None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:52.935873 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f896bbf-08d1-4dc6-966a-904ec5a824fd None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:52.936040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f896bbf-08d1-4dc6-966a-904ec5a824fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:52.956431 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2f896bbf-08d1-4dc6-966a-904ec5a824fd None None] MFA Rules not processed for user `fa0e659eee5e4a9da8d3d7337cb73798`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:30:52.957624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2f896bbf-08d1-4dc6-966a-904ec5a824fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:52.966764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 787/1520] 10.222.0.18 () {58 vars in 1058 bytes} [Mon Aug 29 20:30:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:30:52.974365 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26009aa-c86e-4ad9-81a7-91ad4d58a4d9 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:52.988050 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26009aa-c86e-4ad9-81a7-91ad4d58a4d9 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:52.993644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a3f30-5c9c-46b1-bf57-ddf51fcfc914 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0e659eee5e4a9da8d3d7337cb73798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:52.994280 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a3f30-5c9c-46b1-bf57-ddf51fcfc914 service placement] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:52.994557 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a3f30-5c9c-46b1-bf57-ddf51fcfc914 service placement] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:52.994825 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a3f30-5c9c-46b1-bf57-ddf51fcfc914 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:53.003646 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a3f30-5c9c-46b1-bf57-ddf51fcfc914 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:53.005259 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a3f30-5c9c-46b1-bf57-ddf51fcfc914 service placement] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:53.015449 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 734/1521] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:30:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:30:58.536448 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cfa90-0de8-4cd6-b959-cfb979abb05e None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:58.536984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cfa90-0de8-4cd6-b959-cfb979abb05e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:58.536984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cfa90-0de8-4cd6-b959-cfb979abb05e None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:58.539177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 788/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:30:58.543829 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80c654-4215-44e5-b719-4b7816d3768c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:58.544231 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80c654-4215-44e5-b719-4b7816d3768c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:58.544403 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80c654-4215-44e5-b719-4b7816d3768c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:58.563719 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-dd80c654-4215-44e5-b719-4b7816d3768c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:30:58.564960 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-dd80c654-4215-44e5-b719-4b7816d3768c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:58.572869 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 735/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:30:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:30:58.748328 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442cfa90-0de8-4cd6-b959-cfb979abb05e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:58.763913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442cfa90-0de8-4cd6-b959-cfb979abb05e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:58.769515 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3083a0-609d-45d5-be56-a25fe215bf35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:58.770160 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3083a0-609d-45d5-be56-a25fe215bf35 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:58.770444 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3083a0-609d-45d5-be56-a25fe215bf35 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:58.770721 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3083a0-609d-45d5-be56-a25fe215bf35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:58.779871 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3083a0-609d-45d5-be56-a25fe215bf35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:58.781478 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3083a0-609d-45d5-be56-a25fe215bf35 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:58.790958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 789/1524] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:30:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:30:59.626439 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01679d5-893b-45ae-8434-043d392958b6 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:59.626991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01679d5-893b-45ae-8434-043d392958b6 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:59.626991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01679d5-893b-45ae-8434-043d392958b6 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:59.628339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 736/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 29 20:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:30:59.633188 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19d21a-e7d1-4203-a261-a77abd4aa1cc None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:59.633663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19d21a-e7d1-4203-a261-a77abd4aa1cc None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:59.633771 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19d21a-e7d1-4203-a261-a77abd4aa1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:59.653362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-3e19d21a-e7d1-4203-a261-a77abd4aa1cc None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:30:59.654572 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3e19d21a-e7d1-4203-a261-a77abd4aa1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:30:59.662592 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 790/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 29 20:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:30:59.827411 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01679d5-893b-45ae-8434-043d392958b6 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:30:59.841963 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01679d5-893b-45ae-8434-043d392958b6 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:30:59.847290 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92098170-2d58-41ee-8b6d-0fee1d1660ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:30:59.847876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92098170-2d58-41ee-8b6d-0fee1d1660ce service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:30:59.848175 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92098170-2d58-41ee-8b6d-0fee1d1660ce service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:30:59.848442 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92098170-2d58-41ee-8b6d-0fee1d1660ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:30:59.856982 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92098170-2d58-41ee-8b6d-0fee1d1660ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:30:59.858539 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92098170-2d58-41ee-8b6d-0fee1d1660ce service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:30:59.868322 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 737/1527] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:30:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:31:03.703841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fb76f-7c8c-4a33-b15e-b6726b8fcf6f None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:31:03.704479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fb76f-7c8c-4a33-b15e-b6726b8fcf6f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:31:03.704479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fb76f-7c8c-4a33-b15e-b6726b8fcf6f None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:31:03.705672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 791/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:31:03.711967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70e56e-0a2e-45d8-8a7c-e3efaa09fa90 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:31:03.712450 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70e56e-0a2e-45d8-8a7c-e3efaa09fa90 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:31:03.712744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70e56e-0a2e-45d8-8a7c-e3efaa09fa90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:31:03.733341 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5b70e56e-0a2e-45d8-8a7c-e3efaa09fa90 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:31:03.734547 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5b70e56e-0a2e-45d8-8a7c-e3efaa09fa90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:31:03.742575 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 738/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:31:04.041179 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6fb76f-7c8c-4a33-b15e-b6726b8fcf6f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:31:04.042871 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4c6fb76f-7c8c-4a33-b15e-b6726b8fcf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:31:04.133550 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6fb76f-7c8c-4a33-b15e-b6726b8fcf6f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:31:04.137485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee2c0c-c2bc-4b78-90db-fd6f1b284d87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:31:04.138116 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee2c0c-c2bc-4b78-90db-fd6f1b284d87 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:31:04.138392 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee2c0c-c2bc-4b78-90db-fd6f1b284d87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:31:04.138661 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee2c0c-c2bc-4b78-90db-fd6f1b284d87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:31:04.149007 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee2c0c-c2bc-4b78-90db-fd6f1b284d87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:31:04.150518 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fee2c0c-c2bc-4b78-90db-fd6f1b284d87 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:31:04.160279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 792/1530] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:31:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:31:06.097419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662278c1-1d13-47bc-965c-633261324d46 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:31:06.098071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662278c1-1d13-47bc-965c-633261324d46 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:31:06.098071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662278c1-1d13-47bc-965c-633261324d46 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:31:06.099339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 739/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:31:06.105365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bed301-de16-4114-9b35-b02cd2241180 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:31:06.105797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bed301-de16-4114-9b35-b02cd2241180 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:31:06.105906 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bed301-de16-4114-9b35-b02cd2241180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:31:06.125228 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-99bed301-de16-4114-9b35-b02cd2241180 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:31:06.126417 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-99bed301-de16-4114-9b35-b02cd2241180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:31:06.134669 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 793/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:31:06.422393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662278c1-1d13-47bc-965c-633261324d46 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:31:06.433235 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662278c1-1d13-47bc-965c-633261324d46 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:31:06.436836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d641210-7b8e-4a5c-aca9-b49763890d15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:31:06.437428 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d641210-7b8e-4a5c-aca9-b49763890d15 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:31:06.437707 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d641210-7b8e-4a5c-aca9-b49763890d15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:31:06.437980 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d641210-7b8e-4a5c-aca9-b49763890d15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:31:06.447346 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d641210-7b8e-4a5c-aca9-b49763890d15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:31:06.448806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d641210-7b8e-4a5c-aca9-b49763890d15 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:31:06.457978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 740/1533] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:31:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:31:10.081289 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa15d1-2ae3-40ef-b9d2-315428e06a48 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:31:10.082135 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa15d1-2ae3-40ef-b9d2-315428e06a48 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:31:10.082310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa15d1-2ae3-40ef-b9d2-315428e06a48 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:31:10.083814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 794/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:31:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:31:10.089600 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da639f4b-08a5-482a-912e-ad7c46d512b7 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:31:10.090111 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da639f4b-08a5-482a-912e-ad7c46d512b7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:31:10.090228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da639f4b-08a5-482a-912e-ad7c46d512b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:31:10.098119 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=81 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:31:10.111024 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-da639f4b-08a5-482a-912e-ad7c46d512b7 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:31:10.112261 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-da639f4b-08a5-482a-912e-ad7c46d512b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:31:10.120235 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 741/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:31:10.688254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa15d1-2ae3-40ef-b9d2-315428e06a48 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:31:10.689935 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c6aa15d1-2ae3-40ef-b9d2-315428e06a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:31:10.757351 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:31:10.770955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa15d1-2ae3-40ef-b9d2-315428e06a48 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:31:10.774849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebaf3e7-5870-4384-8abe-cb05b248bcd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:31:10.775458 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebaf3e7-5870-4384-8abe-cb05b248bcd4 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:31:10.775739 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebaf3e7-5870-4384-8abe-cb05b248bcd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:31:10.776042 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebaf3e7-5870-4384-8abe-cb05b248bcd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:31:10.785087 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebaf3e7-5870-4384-8abe-cb05b248bcd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:31:10.786513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebaf3e7-5870-4384-8abe-cb05b248bcd4 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:31:10.797216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 795/1536] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:31:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:31:12.778574 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1de62d-24e6-41ca-ab76-1f9b5a5c132d None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:31:12.779142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1de62d-24e6-41ca-ab76-1f9b5a5c132d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:31:12.779142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1de62d-24e6-41ca-ab76-1f9b5a5c132d None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:31:12.780532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 742/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 29 20:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:31:12.785722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99de66-6512-4171-aa77-1f48a18b334a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:31:12.786201 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99de66-6512-4171-aa77-1f48a18b334a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:31:12.786316 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99de66-6512-4171-aa77-1f48a18b334a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:31:12.806784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4a99de66-6512-4171-aa77-1f48a18b334a None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:31:12.808005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4a99de66-6512-4171-aa77-1f48a18b334a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:31:12.815840 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 796/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 29 20:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:31:13.379020 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1de62d-24e6-41ca-ab76-1f9b5a5c132d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:31:13.381595 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5b1de62d-24e6-41ca-ab76-1f9b5a5c132d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:31:13.394606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1de62d-24e6-41ca-ab76-1f9b5a5c132d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:31:13.398431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde3a252-ba8f-46c5-93c5-85ff16be0a7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:31:13.399026 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3a252-ba8f-46c5-93c5-85ff16be0a7e service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:31:13.399301 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3a252-ba8f-46c5-93c5-85ff16be0a7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:31:13.399568 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3a252-ba8f-46c5-93c5-85ff16be0a7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:31:13.408402 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde3a252-ba8f-46c5-93c5-85ff16be0a7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:31:13.409816 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde3a252-ba8f-46c5-93c5-85ff16be0a7e service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:31:13.419053 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 743/1539] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:31:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:31:22.801623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=11 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:31:23.414976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=8 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:43.948479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f498e-64d3-4e5e-afe5-ab48746ea98b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:43.949144 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f498e-64d3-4e5e-afe5-ab48746ea98b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:43.949144 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f498e-64d3-4e5e-afe5-ab48746ea98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:43.986339 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-007f498e-64d3-4e5e-afe5-ab48746ea98b None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:43.987907 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-007f498e-64d3-4e5e-afe5-ab48746ea98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:44.002072 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 797/1540] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:32:44.012049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c39e0-2ffe-4d5f-b6c1-28f715523e48 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.013239 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c39e0-2ffe-4d5f-b6c1-28f715523e48 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.013239 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c39e0-2ffe-4d5f-b6c1-28f715523e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.040510 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-a75c39e0-2ffe-4d5f-b6c1-28f715523e48 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:44.041862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a75c39e0-2ffe-4d5f-b6c1-28f715523e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:44.053062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 744/1541] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:32:44.058436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f498e-64d3-4e5e-afe5-ab48746ea98b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:44.069869 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f498e-64d3-4e5e-afe5-ab48746ea98b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:44.083300 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d46cba-ab04-4088-8bf5-db1cdb23c152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:44.083954 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d46cba-ab04-4088-8bf5-db1cdb23c152 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.084242 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d46cba-ab04-4088-8bf5-db1cdb23c152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.084516 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d46cba-ab04-4088-8bf5-db1cdb23c152 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.085189 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d46cba-ab04-4088-8bf5-db1cdb23c152 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:32:44.085499 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d46cba-ab04-4088-8bf5-db1cdb23c152 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:44.087321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d46cba-ab04-4088-8bf5-db1cdb23c152 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:44.105978 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 798/1542] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:32:44.114588 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c39e0-2ffe-4d5f-b6c1-28f715523e48 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:44.126962 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c39e0-2ffe-4d5f-b6c1-28f715523e48 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:44.135444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe4463e-cc9b-497a-88fc-b9b16666d9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:44.136090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4463e-cc9b-497a-88fc-b9b16666d9c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.136373 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4463e-cc9b-497a-88fc-b9b16666d9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.136650 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe4463e-cc9b-497a-88fc-b9b16666d9c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.137844 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4463e-cc9b-497a-88fc-b9b16666d9c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:44.139776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe4463e-cc9b-497a-88fc-b9b16666d9c7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:44.196106 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 745/1543] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:32:44 2022] POST /identity/v3/projects => generated 367 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:44.201036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d46cba-ab04-4088-8bf5-db1cdb23c152 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:44.212283 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d46cba-ab04-4088-8bf5-db1cdb23c152 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:44.282528 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d933304-55b5-4b6e-aced-ba4d62687d92 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.282921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d933304-55b5-4b6e-aced-ba4d62687d92 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.283299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d933304-55b5-4b6e-aced-ba4d62687d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.316706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d2e6d-a60c-4210-8d28-23875a6a4fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:44.317348 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d2e6d-a60c-4210-8d28-23875a6a4fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.317633 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d2e6d-a60c-4210-8d28-23875a6a4fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.317909 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d2e6d-a60c-4210-8d28-23875a6a4fb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.318573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d2e6d-a60c-4210-8d28-23875a6a4fb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:44.320558 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d2e6d-a60c-4210-8d28-23875a6a4fb3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:44.324094 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-4d933304-55b5-4b6e-aced-ba4d62687d92 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:44.327831 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4d933304-55b5-4b6e-aced-ba4d62687d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:44.351851 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 746/1544] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:32:44.353467 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d076e37-4caf-4cb6-baed-e9e6fcd60981 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.353760 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d076e37-4caf-4cb6-baed-e9e6fcd60981 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.354035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d076e37-4caf-4cb6-baed-e9e6fcd60981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.369497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 799/1545] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:32:44 2022] POST /identity/v3/users => generated 284 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:44.374070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c3dfb-e795-49bc-8309-c785048bdad4 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.374670 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c3dfb-e795-49bc-8309-c785048bdad4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.375142 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c3dfb-e795-49bc-8309-c785048bdad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.396027 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-0d076e37-4caf-4cb6-baed-e9e6fcd60981 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:44.396808 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0d076e37-4caf-4cb6-baed-e9e6fcd60981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:44.416132 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 747/1546] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:44.417012 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d076e37-4caf-4cb6-baed-e9e6fcd60981 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:44.418386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f61c3dfb-e795-49bc-8309-c785048bdad4 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:44.422551 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f61c3dfb-e795-49bc-8309-c785048bdad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:44.442812 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d076e37-4caf-4cb6-baed-e9e6fcd60981 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:44.444106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 800/1547] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:44.445747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1c556-dad5-4a26-9f88-19edd52a1274 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.447054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1c556-dad5-4a26-9f88-19edd52a1274 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.447966 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1c556-dad5-4a26-9f88-19edd52a1274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.456319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ca4fe-6969-443a-af17-50e125cf0a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:44.457340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ca4fe-6969-443a-af17-50e125cf0a6c admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.457820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ca4fe-6969-443a-af17-50e125cf0a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.458290 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ca4fe-6969-443a-af17-50e125cf0a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.459119 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ca4fe-6969-443a-af17-50e125cf0a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:44.461385 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ca4fe-6969-443a-af17-50e125cf0a6c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:44.473421 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 748/1548] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:44.475028 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1aa8f-69b4-4472-8bc8-4e3ac01e8ad3 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.475515 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1aa8f-69b4-4472-8bc8-4e3ac01e8ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.476014 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1aa8f-69b4-4472-8bc8-4e3ac01e8ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.479640 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-e0e1c556-dad5-4a26-9f88-19edd52a1274 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:44.481990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e0e1c556-dad5-4a26-9f88-19edd52a1274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:44.491046 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 801/1549] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:44.492451 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e724f-5915-4384-a700-14c3c941567c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.492800 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e724f-5915-4384-a700-14c3c941567c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.493145 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e724f-5915-4384-a700-14c3c941567c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.500769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5cc1aa8f-69b4-4472-8bc8-4e3ac01e8ad3 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:44.502507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5cc1aa8f-69b4-4472-8bc8-4e3ac01e8ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:44.512882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 749/1550] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:44.519963 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb30063-636a-439b-ad87-ca5c151cc2ac None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.520199 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a35e724f-5915-4384-a700-14c3c941567c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:44.520199 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a35e724f-5915-4384-a700-14c3c941567c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:44.520801 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb30063-636a-439b-ad87-ca5c151cc2ac None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.522062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb30063-636a-439b-ad87-ca5c151cc2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.532610 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 802/1551] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:44.534579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e724f-5915-4384-a700-14c3c941567c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:44.551188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-bfb30063-636a-439b-ad87-ca5c151cc2ac None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:44.551976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e724f-5915-4384-a700-14c3c941567c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:44.552443 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bfb30063-636a-439b-ad87-ca5c151cc2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:44.557705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda51ad5-5a9f-4972-936c-7a207b3d03b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:44.558554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda51ad5-5a9f-4972-936c-7a207b3d03b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.559309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda51ad5-5a9f-4972-936c-7a207b3d03b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.560311 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda51ad5-5a9f-4972-936c-7a207b3d03b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.561250 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda51ad5-5a9f-4972-936c-7a207b3d03b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:32:44.561663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda51ad5-5a9f-4972-936c-7a207b3d03b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:44.566225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 750/1552] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:44.566225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb30063-636a-439b-ad87-ca5c151cc2ac None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:44.566525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda51ad5-5a9f-4972-936c-7a207b3d03b2 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:44.573188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb30063-636a-439b-ad87-ca5c151cc2ac None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:44.583980 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 803/1553] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:32:44.583980 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062b3a1-ba0b-4729-8d9c-ab6d4a7a0982 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.583980 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062b3a1-ba0b-4729-8d9c-ab6d4a7a0982 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.583980 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062b3a1-ba0b-4729-8d9c-ab6d4a7a0982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.586343 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc85f3-f2e6-43e4-add5-5688e559d6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:44.586984 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc85f3-f2e6-43e4-add5-5688e559d6f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.587266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc85f3-f2e6-43e4-add5-5688e559d6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.587542 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc85f3-f2e6-43e4-add5-5688e559d6f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.588735 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc85f3-f2e6-43e4-add5-5688e559d6f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:44.589835 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc85f3-f2e6-43e4-add5-5688e559d6f9 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:44.599570 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-e062b3a1-ba0b-4729-8d9c-ab6d4a7a0982 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:44.603859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 751/1554] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:44.604172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e062b3a1-ba0b-4729-8d9c-ab6d4a7a0982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:44.604317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc85f3-f2e6-43e4-add5-5688e559d6f9 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:44.614670 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc85f3-f2e6-43e4-add5-5688e559d6f9 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:44.618545 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 804/1555] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:44.620047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338716c0-60df-4197-a65e-9ebdf94ca5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:44.620719 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338716c0-60df-4197-a65e-9ebdf94ca5a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.621061 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338716c0-60df-4197-a65e-9ebdf94ca5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.621374 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338716c0-60df-4197-a65e-9ebdf94ca5a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.622866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338716c0-60df-4197-a65e-9ebdf94ca5a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:32:44.623246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338716c0-60df-4197-a65e-9ebdf94ca5a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:44.624706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb906f42-8539-4a83-95d6-13fe1b21dc8f None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.624858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: WARNING py.warnings [None req-338716c0-60df-4197-a65e-9ebdf94ca5a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:32:44.624858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: warnings.warn(msg) Aug 29 20:32:44.624858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: Aug 29 20:32:44.625125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb906f42-8539-4a83-95d6-13fe1b21dc8f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.625125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb906f42-8539-4a83-95d6-13fe1b21dc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.625623 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338716c0-60df-4197-a65e-9ebdf94ca5a1 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:44.640901 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 752/1556] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:32:44.642248 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd81aa2-18d4-46e0-be63-b1bb867a5c81 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.642609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd81aa2-18d4-46e0-be63-b1bb867a5c81 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.642949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd81aa2-18d4-46e0-be63-b1bb867a5c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.649581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-eb906f42-8539-4a83-95d6-13fe1b21dc8f None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:44.650810 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-eb906f42-8539-4a83-95d6-13fe1b21dc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:44.658771 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 805/1557] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:44.660059 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750dacf-d844-436c-ba45-fe851288c858 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.660351 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750dacf-d844-436c-ba45-fe851288c858 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.660629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750dacf-d844-436c-ba45-fe851288c858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.665087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-efd81aa2-18d4-46e0-be63-b1bb867a5c81 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:44.666413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-efd81aa2-18d4-46e0-be63-b1bb867a5c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:44.674970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 753/1558] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:44.676035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd81aa2-18d4-46e0-be63-b1bb867a5c81 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:44.680879 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-1750dacf-d844-436c-ba45-fe851288c858 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:44.682352 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1750dacf-d844-436c-ba45-fe851288c858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:44.686915 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd81aa2-18d4-46e0-be63-b1bb867a5c81 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:44.691211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 806/1559] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:44.691806 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750dacf-d844-436c-ba45-fe851288c858 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:44.695865 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0111c865-c500-4fad-bc35-743b4a55208d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:44.695865 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111c865-c500-4fad-bc35-743b4a55208d admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.695865 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111c865-c500-4fad-bc35-743b4a55208d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.695865 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111c865-c500-4fad-bc35-743b4a55208d admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.696529 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0111c865-c500-4fad-bc35-743b4a55208d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:44.696529 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0111c865-c500-4fad-bc35-743b4a55208d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:44.701508 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750dacf-d844-436c-ba45-fe851288c858 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:44.706161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d88cc5-b168-4ab3-9823-6f32e2b41156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:44.706892 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d88cc5-b168-4ab3-9823-6f32e2b41156 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.707124 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d88cc5-b168-4ab3-9823-6f32e2b41156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.707405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d88cc5-b168-4ab3-9823-6f32e2b41156 admin admin] PATH_INFO: `/v3/projects/c6a65050f39e49349b0eab86031bab9f/users/18b635c193494d43adc087175b4eed4d/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.725914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d88cc5-b168-4ab3-9823-6f32e2b41156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a65050f39e49349b0eab86031bab9f, user_id=18b635c193494d43adc087175b4eed4d, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:44.726994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: WARNING py.warnings [None req-87d88cc5-b168-4ab3-9823-6f32e2b41156 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:32:44.726994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: warnings.warn(msg) Aug 29 20:32:44.726994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: Aug 29 20:32:44.731049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 754/1560] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:32:44 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:44.731049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0111c865-c500-4fad-bc35-743b4a55208d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:44.731250 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d88cc5-b168-4ab3-9823-6f32e2b41156 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:44.744637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 807/1561] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:32:44 2022] PUT /identity/v3/projects/c6a65050f39e49349b0eab86031bab9f/users/18b635c193494d43adc087175b4eed4d/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:44.745921 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2b9d2-165c-498d-9444-cbf5c9e5b2fc None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.746206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2b9d2-165c-498d-9444-cbf5c9e5b2fc None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.746495 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2b9d2-165c-498d-9444-cbf5c9e5b2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.777866 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-e6d2b9d2-165c-498d-9444-cbf5c9e5b2fc None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:44.779609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e6d2b9d2-165c-498d-9444-cbf5c9e5b2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:44.801900 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0111c865-c500-4fad-bc35-743b4a55208d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:44.834127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:44.843766 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:44.855652 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 808/1562] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:44.857809 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2b9d2-165c-498d-9444-cbf5c9e5b2fc None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:44.870447 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224a554-ab77-43a8-8018-ec3a7841f357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:44.871157 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224a554-ab77-43a8-8018-ec3a7841f357 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.871522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224a554-ab77-43a8-8018-ec3a7841f357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.871851 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224a554-ab77-43a8-8018-ec3a7841f357 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.872649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224a554-ab77-43a8-8018-ec3a7841f357 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:32:44.873024 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224a554-ab77-43a8-8018-ec3a7841f357 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:44.873576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2b9d2-165c-498d-9444-cbf5c9e5b2fc None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:44.874784 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224a554-ab77-43a8-8018-ec3a7841f357 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:44.881230 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a592b-0a21-4a1c-b6e6-adebfa4349e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:44.882314 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a592b-0a21-4a1c-b6e6-adebfa4349e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.882842 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a592b-0a21-4a1c-b6e6-adebfa4349e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.883307 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a592b-0a21-4a1c-b6e6-adebfa4349e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.884437 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a592b-0a21-4a1c-b6e6-adebfa4349e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:44.886219 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: WARNING py.warnings [None req-d30a592b-0a21-4a1c-b6e6-adebfa4349e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:32:44.886219 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: warnings.warn(msg) Aug 29 20:32:44.886219 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: Aug 29 20:32:44.894812 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 755/1563] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:32:44.894812 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224a554-ab77-43a8-8018-ec3a7841f357 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:44.895121 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a592b-0a21-4a1c-b6e6-adebfa4349e4 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:44.900483 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224a554-ab77-43a8-8018-ec3a7841f357 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:44.906635 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa184bb-fee6-46b1-9a19-dc456329e376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:44.907422 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa184bb-fee6-46b1-9a19-dc456329e376 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.907694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa184bb-fee6-46b1-9a19-dc456329e376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.908004 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa184bb-fee6-46b1-9a19-dc456329e376 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.908652 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa184bb-fee6-46b1-9a19-dc456329e376 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:32:44.909325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa184bb-fee6-46b1-9a19-dc456329e376 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:44.910552 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa184bb-fee6-46b1-9a19-dc456329e376 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:44.924122 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 756/1564] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:32:44.925465 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd3c1b-a9c3-4e1e-a249-2f9eb53d3f7e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:44.925688 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd3c1b-a9c3-4e1e-a249-2f9eb53d3f7e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:44.925946 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd3c1b-a9c3-4e1e-a249-2f9eb53d3f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:44.945061 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 809/1565] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:32:44 2022] POST /identity/v3/projects => generated 379 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:44.945061 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a592b-0a21-4a1c-b6e6-adebfa4349e4 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:44.949796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-8acd3c1b-a9c3-4e1e-a249-2f9eb53d3f7e None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:44.951539 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8acd3c1b-a9c3-4e1e-a249-2f9eb53d3f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:45.008979 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a592b-0a21-4a1c-b6e6-adebfa4349e4 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.014188 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996208fa-ac7d-446d-9f16-eb4e6fe4f235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.014805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996208fa-ac7d-446d-9f16-eb4e6fe4f235 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.015087 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996208fa-ac7d-446d-9f16-eb4e6fe4f235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.015363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996208fa-ac7d-446d-9f16-eb4e6fe4f235 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.016048 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996208fa-ac7d-446d-9f16-eb4e6fe4f235 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:32:45.016357 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996208fa-ac7d-446d-9f16-eb4e6fe4f235 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.016735 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 757/1566] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:45.017572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd3c1b-a9c3-4e1e-a249-2f9eb53d3f7e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.017977 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996208fa-ac7d-446d-9f16-eb4e6fe4f235 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.027577 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd3c1b-a9c3-4e1e-a249-2f9eb53d3f7e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.035726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 810/1567] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:32:45.035726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2372404-7a8f-4e9c-9ce6-09dd17c7fb1d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.035726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2372404-7a8f-4e9c-9ce6-09dd17c7fb1d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.035726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2372404-7a8f-4e9c-9ce6-09dd17c7fb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.038081 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4269f8-611a-478f-95ca-b41408e832d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.038713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4269f8-611a-478f-95ca-b41408e832d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.038995 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4269f8-611a-478f-95ca-b41408e832d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.039529 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4269f8-611a-478f-95ca-b41408e832d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.040439 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4269f8-611a-478f-95ca-b41408e832d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.041533 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: WARNING py.warnings [None req-9a4269f8-611a-478f-95ca-b41408e832d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:32:45.041533 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: warnings.warn(msg) Aug 29 20:32:45.041533 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: Aug 29 20:32:45.042845 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4269f8-611a-478f-95ca-b41408e832d9 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.063825 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-c2372404-7a8f-4e9c-9ce6-09dd17c7fb1d None None] MFA Rules not processed for user `18b635c193494d43adc087175b4eed4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:45.065158 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c2372404-7a8f-4e9c-9ce6-09dd17c7fb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:45.069686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 758/1568] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:32:45 2022] POST /identity/v3/users => generated 291 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:45.069686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf572a7f-02ad-476b-b41f-de16ae678ed3 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.069686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf572a7f-02ad-476b-b41f-de16ae678ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.069686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf572a7f-02ad-476b-b41f-de16ae678ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.088959 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-cf572a7f-02ad-476b-b41f-de16ae678ed3 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:45.090516 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-cf572a7f-02ad-476b-b41f-de16ae678ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:45.099890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 759/1569] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:45.101203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf572a7f-02ad-476b-b41f-de16ae678ed3 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.116116 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf572a7f-02ad-476b-b41f-de16ae678ed3 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.122784 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e1cb2-84be-4b18-8a78-e9e5c059ef3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.123441 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e1cb2-84be-4b18-8a78-e9e5c059ef3f admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.123716 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e1cb2-84be-4b18-8a78-e9e5c059ef3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.124030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e1cb2-84be-4b18-8a78-e9e5c059ef3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.124720 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e1cb2-84be-4b18-8a78-e9e5c059ef3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.126786 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e1cb2-84be-4b18-8a78-e9e5c059ef3f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.139720 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 811/1570] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:45.140625 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2372404-7a8f-4e9c-9ce6-09dd17c7fb1d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.152642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2372404-7a8f-4e9c-9ce6-09dd17c7fb1d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.158137 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f780a-5d37-40ab-9140-4727a2468ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.158888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f780a-5d37-40ab-9140-4727a2468ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.159168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f780a-5d37-40ab-9140-4727a2468ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.159447 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f780a-5d37-40ab-9140-4727a2468ba6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.160159 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f780a-5d37-40ab-9140-4727a2468ba6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.162083 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f780a-5d37-40ab-9140-4727a2468ba6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.164747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 760/1571] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:32:45 2022] POST /identity/v3/projects => generated 393 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:45.165690 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e1cb2-84be-4b18-8a78-e9e5c059ef3f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.191476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 812/1572] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:32:45 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:45.192682 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f780a-5d37-40ab-9140-4727a2468ba6 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.229817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:45.234674 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e1cb2-84be-4b18-8a78-e9e5c059ef3f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.254259 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:45.257670 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f780a-5d37-40ab-9140-4727a2468ba6 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.262881 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec9d83-dfc0-4657-a591-c1da4cecbe13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.263447 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec9d83-dfc0-4657-a591-c1da4cecbe13 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.263733 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec9d83-dfc0-4657-a591-c1da4cecbe13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.264040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec9d83-dfc0-4657-a591-c1da4cecbe13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.264724 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec9d83-dfc0-4657-a591-c1da4cecbe13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:32:45.265029 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec9d83-dfc0-4657-a591-c1da4cecbe13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.266645 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec9d83-dfc0-4657-a591-c1da4cecbe13 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.280107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 813/1573] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:32:45.281020 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec9d83-dfc0-4657-a591-c1da4cecbe13 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.292280 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec9d83-dfc0-4657-a591-c1da4cecbe13 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.297069 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76345216-0128-4aa8-81cb-314aa4821c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.297722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76345216-0128-4aa8-81cb-314aa4821c4a admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.298006 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76345216-0128-4aa8-81cb-314aa4821c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.298289 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76345216-0128-4aa8-81cb-314aa4821c4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.298930 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76345216-0128-4aa8-81cb-314aa4821c4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.303955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76345216-0128-4aa8-81cb-314aa4821c4a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.308618 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585c46d-0ffa-4802-9a2d-d9d9d48f8684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.309441 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585c46d-0ffa-4802-9a2d-d9d9d48f8684 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.309732 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585c46d-0ffa-4802-9a2d-d9d9d48f8684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.309998 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585c46d-0ffa-4802-9a2d-d9d9d48f8684 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.310666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6585c46d-0ffa-4802-9a2d-d9d9d48f8684 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.312653 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6585c46d-0ffa-4802-9a2d-d9d9d48f8684 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.330236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 814/1574] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:32:45 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:45.331116 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76345216-0128-4aa8-81cb-314aa4821c4a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.335769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 761/1575] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:32:45 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:45.336525 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585c46d-0ffa-4802-9a2d-d9d9d48f8684 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.347719 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585c46d-0ffa-4802-9a2d-d9d9d48f8684 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.401704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76345216-0128-4aa8-81cb-314aa4821c4a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.407303 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccaafd3-5a74-4273-a359-bf97bcd86115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.407958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccaafd3-5a74-4273-a359-bf97bcd86115 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.408242 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccaafd3-5a74-4273-a359-bf97bcd86115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.408519 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccaafd3-5a74-4273-a359-bf97bcd86115 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.409399 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccaafd3-5a74-4273-a359-bf97bcd86115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.411048 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccaafd3-5a74-4273-a359-bf97bcd86115 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.411841 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66282e49-e1ee-4983-82ec-a82fa5e83190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.412620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66282e49-e1ee-4983-82ec-a82fa5e83190 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.412904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66282e49-e1ee-4983-82ec-a82fa5e83190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.413181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66282e49-e1ee-4983-82ec-a82fa5e83190 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.413847 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66282e49-e1ee-4983-82ec-a82fa5e83190 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:32:45.414498 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66282e49-e1ee-4983-82ec-a82fa5e83190 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.416169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66282e49-e1ee-4983-82ec-a82fa5e83190 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.421712 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 815/1576] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:45.422587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccaafd3-5a74-4273-a359-bf97bcd86115 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.429387 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 762/1577] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:32:45.430234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66282e49-e1ee-4983-82ec-a82fa5e83190 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.441921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66282e49-e1ee-4983-82ec-a82fa5e83190 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.448152 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a68b3e-e665-4079-b7d2-58274cf11a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.448771 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a68b3e-e665-4079-b7d2-58274cf11a1c admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.449049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a68b3e-e665-4079-b7d2-58274cf11a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.449324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a68b3e-e665-4079-b7d2-58274cf11a1c admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.450490 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a68b3e-e665-4079-b7d2-58274cf11a1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.452459 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a68b3e-e665-4079-b7d2-58274cf11a1c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.479225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 763/1578] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:32:45 2022] POST /identity/v3/users => generated 297 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:45.479225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a68b3e-e665-4079-b7d2-58274cf11a1c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.484039 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccaafd3-5a74-4273-a359-bf97bcd86115 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.487104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a68b3e-e665-4079-b7d2-58274cf11a1c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.488140 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c22fd-eb5a-4e6c-be5e-ce811086c82a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.488838 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c22fd-eb5a-4e6c-be5e-ce811086c82a service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.489122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c22fd-eb5a-4e6c-be5e-ce811086c82a service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.489401 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c22fd-eb5a-4e6c-be5e-ce811086c82a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.491996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603197f-e4e5-4aea-926f-80841c2e1cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.492673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603197f-e4e5-4aea-926f-80841c2e1cee admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.492950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603197f-e4e5-4aea-926f-80841c2e1cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.493287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603197f-e4e5-4aea-926f-80841c2e1cee admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.493914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603197f-e4e5-4aea-926f-80841c2e1cee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.495886 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603197f-e4e5-4aea-926f-80841c2e1cee admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.519142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 764/1579] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:32:45 2022] POST /identity/v3/users => generated 288 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:45.519865 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603197f-e4e5-4aea-926f-80841c2e1cee admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.531206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603197f-e4e5-4aea-926f-80841c2e1cee admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.535799 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81be478-d867-4f2f-b1eb-b0cf16aa3cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.536431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81be478-d867-4f2f-b1eb-b0cf16aa3cbf admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.536706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81be478-d867-4f2f-b1eb-b0cf16aa3cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.536975 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81be478-d867-4f2f-b1eb-b0cf16aa3cbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.537596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81be478-d867-4f2f-b1eb-b0cf16aa3cbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.539744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81be478-d867-4f2f-b1eb-b0cf16aa3cbf admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.551922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:45.554661 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c22fd-eb5a-4e6c-be5e-ce811086c82a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.556848 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c22fd-eb5a-4e6c-be5e-ce811086c82a service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.569078 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 816/1580] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:32:45.569738 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c22fd-eb5a-4e6c-be5e-ce811086c82a service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.573399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 765/1581] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:32:45 2022] POST /identity/v3/projects => generated 403 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:45.574409 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81be478-d867-4f2f-b1eb-b0cf16aa3cbf admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.581289 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c22fd-eb5a-4e6c-be5e-ce811086c82a service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.627830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:45.645355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cf592-8c68-46cb-94fb-d05e5ec4a537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.645769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81be478-d867-4f2f-b1eb-b0cf16aa3cbf admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.645976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cf592-8c68-46cb-94fb-d05e5ec4a537 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.646259 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cf592-8c68-46cb-94fb-d05e5ec4a537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.646527 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5cf592-8c68-46cb-94fb-d05e5ec4a537 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.647163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5cf592-8c68-46cb-94fb-d05e5ec4a537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.649305 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5cf592-8c68-46cb-94fb-d05e5ec4a537 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.651415 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40199142-1d3d-4fbe-b4e7-49e13faeb403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.652270 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40199142-1d3d-4fbe-b4e7-49e13faeb403 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.652881 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40199142-1d3d-4fbe-b4e7-49e13faeb403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.653427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40199142-1d3d-4fbe-b4e7-49e13faeb403 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.654174 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40199142-1d3d-4fbe-b4e7-49e13faeb403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.656151 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40199142-1d3d-4fbe-b4e7-49e13faeb403 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.670791 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 766/1582] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:45.671866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40199142-1d3d-4fbe-b4e7-49e13faeb403 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.678231 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 817/1583] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:32:45 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:45.679308 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cf592-8c68-46cb-94fb-d05e5ec4a537 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.686851 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40199142-1d3d-4fbe-b4e7-49e13faeb403 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.691327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5cf592-8c68-46cb-94fb-d05e5ec4a537 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.692311 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abacdc4-4fbe-49f0-83de-4fe01cd1e930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.692950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abacdc4-4fbe-49f0-83de-4fe01cd1e930 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.693267 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abacdc4-4fbe-49f0-83de-4fe01cd1e930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.693549 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abacdc4-4fbe-49f0-83de-4fe01cd1e930 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.694246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abacdc4-4fbe-49f0-83de-4fe01cd1e930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.695896 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abacdc4-4fbe-49f0-83de-4fe01cd1e930 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.697781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80593b4-d502-4eb4-8576-e99776c79072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.698408 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80593b4-d502-4eb4-8576-e99776c79072 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.698689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80593b4-d502-4eb4-8576-e99776c79072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.698995 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80593b4-d502-4eb4-8576-e99776c79072 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.699643 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80593b4-d502-4eb4-8576-e99776c79072 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.702145 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80593b4-d502-4eb4-8576-e99776c79072 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.708703 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 767/1584] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:45.712783 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abacdc4-4fbe-49f0-83de-4fe01cd1e930 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.725019 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abacdc4-4fbe-49f0-83de-4fe01cd1e930 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.730015 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9cd764-a8cb-4c47-9f6f-015b24c225ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.730668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9cd764-a8cb-4c47-9f6f-015b24c225ee admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.730924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9cd764-a8cb-4c47-9f6f-015b24c225ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.731216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9cd764-a8cb-4c47-9f6f-015b24c225ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.732045 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9cd764-a8cb-4c47-9f6f-015b24c225ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.733512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9cd764-a8cb-4c47-9f6f-015b24c225ee admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.743927 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 818/1585] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:32:45 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:45.743927 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80593b4-d502-4eb4-8576-e99776c79072 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.749551 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 768/1586] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:45.749551 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9cd764-a8cb-4c47-9f6f-015b24c225ee admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.760547 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9cd764-a8cb-4c47-9f6f-015b24c225ee admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.819542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80593b4-d502-4eb4-8576-e99776c79072 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.824598 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdf9aa-dfc6-4107-80b1-cec076ddcc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.825767 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdf9aa-dfc6-4107-80b1-cec076ddcc28 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.826194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdf9aa-dfc6-4107-80b1-cec076ddcc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.826472 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdf9aa-dfc6-4107-80b1-cec076ddcc28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.827129 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebdf9aa-dfc6-4107-80b1-cec076ddcc28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.828941 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebdf9aa-dfc6-4107-80b1-cec076ddcc28 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.829140 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f13084-b637-4934-b61f-0f03e2f1f4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.830385 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f13084-b637-4934-b61f-0f03e2f1f4b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.830738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f13084-b637-4934-b61f-0f03e2f1f4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.831213 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f13084-b637-4934-b61f-0f03e2f1f4b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.831963 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f13084-b637-4934-b61f-0f03e2f1f4b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.834468 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f13084-b637-4934-b61f-0f03e2f1f4b6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.843581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 819/1587] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:45.843581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdf9aa-dfc6-4107-80b1-cec076ddcc28 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.854683 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdf9aa-dfc6-4107-80b1-cec076ddcc28 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.860555 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4e660-138b-4cf0-bf7f-2a1fd7948216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.861262 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4e660-138b-4cf0-bf7f-2a1fd7948216 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.861379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 769/1588] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:32:45 2022] POST /identity/v3/users => generated 302 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:45.861626 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4e660-138b-4cf0-bf7f-2a1fd7948216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.861977 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4e660-138b-4cf0-bf7f-2a1fd7948216 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.862237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f13084-b637-4934-b61f-0f03e2f1f4b6 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.862754 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4e660-138b-4cf0-bf7f-2a1fd7948216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.864468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4e660-138b-4cf0-bf7f-2a1fd7948216 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.876176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f13084-b637-4934-b61f-0f03e2f1f4b6 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.876508 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 820/1589] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:45.877538 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4e660-138b-4cf0-bf7f-2a1fd7948216 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.884198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d8d19-4fbe-465a-911b-61286dce9eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.884820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d8d19-4fbe-465a-911b-61286dce9eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.885111 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d8d19-4fbe-465a-911b-61286dce9eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.885388 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d8d19-4fbe-465a-911b-61286dce9eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.886041 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d8d19-4fbe-465a-911b-61286dce9eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.887849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d8d19-4fbe-465a-911b-61286dce9eb6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.891156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4e660-138b-4cf0-bf7f-2a1fd7948216 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.896733 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba650e75-936f-48d0-affa-fbe8ded37e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.897475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba650e75-936f-48d0-affa-fbe8ded37e40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.897762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba650e75-936f-48d0-affa-fbe8ded37e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.898040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba650e75-936f-48d0-affa-fbe8ded37e40 admin admin] PATH_INFO: `/v3/projects/20abd4fa69204e62b7a49322b31d958f/users/e5e618cf472e4275bec4d6222fa29ebe/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.900183 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 770/1590] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:45.901026 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d8d19-4fbe-465a-911b-61286dce9eb6 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.915086 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d8d19-4fbe-465a-911b-61286dce9eb6 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.917397 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba650e75-936f-48d0-affa-fbe8ded37e40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20abd4fa69204e62b7a49322b31d958f, user_id=e5e618cf472e4275bec4d6222fa29ebe, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.920878 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba650e75-936f-48d0-affa-fbe8ded37e40 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.921663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d4c39-a5a4-473b-90f0-acca10d44e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:45.922428 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d4c39-a5a4-473b-90f0-acca10d44e08 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:45.922907 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d4c39-a5a4-473b-90f0-acca10d44e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:45.923247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d4c39-a5a4-473b-90f0-acca10d44e08 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:45.923900 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d4c39-a5a4-473b-90f0-acca10d44e08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:45.925945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d4c39-a5a4-473b-90f0-acca10d44e08 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:45.936042 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 821/1591] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:32:45 2022] PUT /identity/v3/projects/20abd4fa69204e62b7a49322b31d958f/users/e5e618cf472e4275bec4d6222fa29ebe/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:45.937277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba650e75-936f-48d0-affa-fbe8ded37e40 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.960546 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 771/1592] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:32:45 2022] POST /identity/v3/users => generated 288 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:45.961598 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d4c39-a5a4-473b-90f0-acca10d44e08 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:45.971724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d4c39-a5a4-473b-90f0-acca10d44e08 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:45.977602 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:46.018812 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba650e75-936f-48d0-affa-fbe8ded37e40 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:46.025635 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1bd05-ba0f-4da5-b69f-21dbce611baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:46.026393 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1bd05-ba0f-4da5-b69f-21dbce611baa admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.026735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1bd05-ba0f-4da5-b69f-21dbce611baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.027102 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1bd05-ba0f-4da5-b69f-21dbce611baa admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.027858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1bd05-ba0f-4da5-b69f-21dbce611baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:46.029633 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1bd05-ba0f-4da5-b69f-21dbce611baa admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:46.040536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c41f8c-7c92-404d-9e8e-80f740c79731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:46.041158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c41f8c-7c92-404d-9e8e-80f740c79731 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.041399 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 822/1593] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:46.041509 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c41f8c-7c92-404d-9e8e-80f740c79731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.041791 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c41f8c-7c92-404d-9e8e-80f740c79731 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.042520 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c41f8c-7c92-404d-9e8e-80f740c79731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:46.044695 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c41f8c-7c92-404d-9e8e-80f740c79731 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:46.045947 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1bd05-ba0f-4da5-b69f-21dbce611baa admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:46.056824 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 772/1594] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:46.057022 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1bd05-ba0f-4da5-b69f-21dbce611baa admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:46.057598 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c41f8c-7c92-404d-9e8e-80f740c79731 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:46.064811 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf412a-c819-49f3-a676-c9da64183fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:46.065439 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf412a-c819-49f3-a676-c9da64183fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.065722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf412a-c819-49f3-a676-c9da64183fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.065993 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf412a-c819-49f3-a676-c9da64183fa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.066979 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf412a-c819-49f3-a676-c9da64183fa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:46.068636 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf412a-c819-49f3-a676-c9da64183fa1 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:46.072387 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c41f8c-7c92-404d-9e8e-80f740c79731 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:46.079310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 823/1595] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:46.080267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf412a-c819-49f3-a676-c9da64183fa1 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:46.084891 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c2865-22c2-4c32-a8e6-a22a0c7bed3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:46.085738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c2865-22c2-4c32-a8e6-a22a0c7bed3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.086371 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c2865-22c2-4c32-a8e6-a22a0c7bed3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.086758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c2865-22c2-4c32-a8e6-a22a0c7bed3e admin admin] PATH_INFO: `/v3/projects/6314d8436170404097713aa7764dff94/users/03117ffa81334df8a3467adbfaf96059/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.091270 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf412a-c819-49f3-a676-c9da64183fa1 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:46.104915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88a70b-d02c-46d5-8185-6a612991bfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:46.105699 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88a70b-d02c-46d5-8185-6a612991bfc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.105896 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88a70b-d02c-46d5-8185-6a612991bfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.106175 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88a70b-d02c-46d5-8185-6a612991bfc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.107411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88a70b-d02c-46d5-8185-6a612991bfc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:46.110410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c2865-22c2-4c32-a8e6-a22a0c7bed3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6314d8436170404097713aa7764dff94, user_id=03117ffa81334df8a3467adbfaf96059, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:46.113368 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c2865-22c2-4c32-a8e6-a22a0c7bed3e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:46.115927 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88a70b-d02c-46d5-8185-6a612991bfc1 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:46.121153 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:46.134835 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 824/1596] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:46.136600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0a3a7-3e89-49d1-9564-5da9a2cf1b4b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.137036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0a3a7-3e89-49d1-9564-5da9a2cf1b4b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.137455 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0a3a7-3e89-49d1-9564-5da9a2cf1b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.137706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 773/1597] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:32:46 2022] PUT /identity/v3/projects/6314d8436170404097713aa7764dff94/users/03117ffa81334df8a3467adbfaf96059/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:46.138595 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c2865-22c2-4c32-a8e6-a22a0c7bed3e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:46.176818 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-59a0a3a7-3e89-49d1-9564-5da9a2cf1b4b None None] MFA Rules not processed for user `e5e618cf472e4275bec4d6222fa29ebe`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:46.178654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-59a0a3a7-3e89-49d1-9564-5da9a2cf1b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:46.224127 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c2865-22c2-4c32-a8e6-a22a0c7bed3e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:46.228821 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b2b21-b765-45b8-bd23-efea27167ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:46.229713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b2b21-b765-45b8-bd23-efea27167ede admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.230241 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b2b21-b765-45b8-bd23-efea27167ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.230602 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b2b21-b765-45b8-bd23-efea27167ede admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.232120 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b2b21-b765-45b8-bd23-efea27167ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:46.234243 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b2b21-b765-45b8-bd23-efea27167ede admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:46.245571 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 774/1598] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:46.246565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b2b21-b765-45b8-bd23-efea27167ede admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:46.258444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b2b21-b765-45b8-bd23-efea27167ede admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:46.263431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc856f-4e7e-45c4-9991-2d3d345fd661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:46.264094 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc856f-4e7e-45c4-9991-2d3d345fd661 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.264812 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc856f-4e7e-45c4-9991-2d3d345fd661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.264812 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc856f-4e7e-45c4-9991-2d3d345fd661 admin admin] PATH_INFO: `/v3/projects/8436d85d931a41cfa6d626c08ad82ef4/users/8fbc5bd64315421e97931402fb0e3a6a/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.275459 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 825/1599] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:46.276354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a0a3a7-3e89-49d1-9564-5da9a2cf1b4b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:46.290845 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc856f-4e7e-45c4-9991-2d3d345fd661 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8436d85d931a41cfa6d626c08ad82ef4, user_id=8fbc5bd64315421e97931402fb0e3a6a, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:46.291584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a0a3a7-3e89-49d1-9564-5da9a2cf1b4b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:46.294823 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc856f-4e7e-45c4-9991-2d3d345fd661 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:46.297814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae21f77-95bf-4df4-99c9-cca0400069f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:46.298561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae21f77-95bf-4df4-99c9-cca0400069f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.299149 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae21f77-95bf-4df4-99c9-cca0400069f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.299431 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae21f77-95bf-4df4-99c9-cca0400069f4 admin admin] PATH_INFO: `/v3/projects/e97ceb2f66cf4762b62ffbedcf11b9a1/users/abefeff1898c4f27b14f11fb8c6ab740/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.324093 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 775/1600] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:32:46 2022] PUT /identity/v3/projects/8436d85d931a41cfa6d626c08ad82ef4/users/8fbc5bd64315421e97931402fb0e3a6a/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:46.325324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc856f-4e7e-45c4-9991-2d3d345fd661 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:46.331717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae21f77-95bf-4df4-99c9-cca0400069f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e97ceb2f66cf4762b62ffbedcf11b9a1, user_id=abefeff1898c4f27b14f11fb8c6ab740, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:46.336074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae21f77-95bf-4df4-99c9-cca0400069f4 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:46.352229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 826/1601] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:32:46 2022] PUT /identity/v3/projects/e97ceb2f66cf4762b62ffbedcf11b9a1/users/abefeff1898c4f27b14f11fb8c6ab740/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:46.353042 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae21f77-95bf-4df4-99c9-cca0400069f4 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:46.398266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc856f-4e7e-45c4-9991-2d3d345fd661 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:46.416487 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae21f77-95bf-4df4-99c9-cca0400069f4 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:46.421763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79280905-906b-4628-a0f0-40dcfdee296b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:46.422511 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79280905-906b-4628-a0f0-40dcfdee296b admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.422794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79280905-906b-4628-a0f0-40dcfdee296b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.423074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79280905-906b-4628-a0f0-40dcfdee296b admin admin] PATH_INFO: `/v3/projects/544c78c5c2c44219a7f236b24a107da4/users/adc5854ed52042369a9b153af7da4cfb/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.431273 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:46.438709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79280905-906b-4628-a0f0-40dcfdee296b admin admin] RBAC: Authorizing `identity:create_grant(project_id=544c78c5c2c44219a7f236b24a107da4, user_id=adc5854ed52042369a9b153af7da4cfb, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:46.441995 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79280905-906b-4628-a0f0-40dcfdee296b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:46.455927 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 827/1602] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:32:46 2022] PUT /identity/v3/projects/544c78c5c2c44219a7f236b24a107da4/users/adc5854ed52042369a9b153af7da4cfb/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:46.457346 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6f607-6ba4-4a6d-971f-580d9e5ea70b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.457634 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6f607-6ba4-4a6d-971f-580d9e5ea70b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.457906 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6f607-6ba4-4a6d-971f-580d9e5ea70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.460312 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46aba60-cd7a-49fd-a777-5677cf8bcb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:46.460930 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46aba60-cd7a-49fd-a777-5677cf8bcb5d admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.461212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46aba60-cd7a-49fd-a777-5677cf8bcb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.461485 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46aba60-cd7a-49fd-a777-5677cf8bcb5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.462165 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46aba60-cd7a-49fd-a777-5677cf8bcb5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:46.463715 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46aba60-cd7a-49fd-a777-5677cf8bcb5d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:46.474506 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 776/1603] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:46.475427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46aba60-cd7a-49fd-a777-5677cf8bcb5d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:46.482577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-0fe6f607-6ba4-4a6d-971f-580d9e5ea70b None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:46.483780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0fe6f607-6ba4-4a6d-971f-580d9e5ea70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:46.485800 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46aba60-cd7a-49fd-a777-5677cf8bcb5d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:46.494266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:46.546073 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 828/1604] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:46.547564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872c9d7-dce5-4e9c-8c30-2fee34d86eeb None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.547852 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872c9d7-dce5-4e9c-8c30-2fee34d86eeb None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.548157 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872c9d7-dce5-4e9c-8c30-2fee34d86eeb None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.549406 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 829/1605] 10.222.0.18 () {54 vars in 962 bytes} [Mon Aug 29 20:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:32:46.550635 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7855f9-af5f-4f45-b507-cb23fe1c16a4 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.550913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7855f9-af5f-4f45-b507-cb23fe1c16a4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.551626 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7855f9-af5f-4f45-b507-cb23fe1c16a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.551725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c2947-3fcc-4c7e-a448-da0eab262de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:46.551961 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c2947-3fcc-4c7e-a448-da0eab262de6 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.552235 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c2947-3fcc-4c7e-a448-da0eab262de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.552507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c2947-3fcc-4c7e-a448-da0eab262de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.553175 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c2947-3fcc-4c7e-a448-da0eab262de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:46.554757 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c2947-3fcc-4c7e-a448-da0eab262de6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:46.565632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 777/1606] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:46.566759 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfe4ab-1155-4151-8ed7-a32f5ac83ee4 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.567042 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfe4ab-1155-4151-8ed7-a32f5ac83ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.567312 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdfe4ab-1155-4151-8ed7-a32f5ac83ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.573144 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4b7855f9-af5f-4f45-b507-cb23fe1c16a4 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:46.574450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4b7855f9-af5f-4f45-b507-cb23fe1c16a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:46.582309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 830/1607] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:46.585285 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65cf76-334e-43d1-966d-0e89a5971ccf None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.585648 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65cf76-334e-43d1-966d-0e89a5971ccf None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.586405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65cf76-334e-43d1-966d-0e89a5971ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.596698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-4fdfe4ab-1155-4151-8ed7-a32f5ac83ee4 None None] MFA Rules not processed for user `abefeff1898c4f27b14f11fb8c6ab740`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:46.597986 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4fdfe4ab-1155-4151-8ed7-a32f5ac83ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:46.605180 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-3c65cf76-334e-43d1-966d-0e89a5971ccf None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:46.606884 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3c65cf76-334e-43d1-966d-0e89a5971ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:46.614421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 831/1608] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:46.615201 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65cf76-334e-43d1-966d-0e89a5971ccf None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:46.625055 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65cf76-334e-43d1-966d-0e89a5971ccf None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:46.630493 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a07eb-691b-4e81-b28c-b2e639d94fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:46.631159 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a07eb-691b-4e81-b28c-b2e639d94fd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.631439 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a07eb-691b-4e81-b28c-b2e639d94fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.631717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a07eb-691b-4e81-b28c-b2e639d94fd8 admin admin] PATH_INFO: `/v3/projects/744393782ba8406c9c1c9f5819696be0/users/b2b25de07f144abcadceb35e2bad5ca2/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.646377 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a07eb-691b-4e81-b28c-b2e639d94fd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=744393782ba8406c9c1c9f5819696be0, user_id=b2b25de07f144abcadceb35e2bad5ca2, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:46.649079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a07eb-691b-4e81-b28c-b2e639d94fd8 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:46.664058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 832/1609] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:32:46 2022] PUT /identity/v3/projects/744393782ba8406c9c1c9f5819696be0/users/b2b25de07f144abcadceb35e2bad5ca2/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:46.665562 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655dcb8-adfe-44f8-a886-54b6deb3c114 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.665847 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655dcb8-adfe-44f8-a886-54b6deb3c114 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.666122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655dcb8-adfe-44f8-a886-54b6deb3c114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.671297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 778/1610] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:46.672188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdfe4ab-1155-4151-8ed7-a32f5ac83ee4 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:46.713541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-e655dcb8-adfe-44f8-a886-54b6deb3c114 None None] MFA Rules not processed for user `c0c44754a1c04a2b87081aba6cb509d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:46.715226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e655dcb8-adfe-44f8-a886-54b6deb3c114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:46.736581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdfe4ab-1155-4151-8ed7-a32f5ac83ee4 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:46.787768 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0660a-3af0-4bb1-9f61-495d8719a607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:46.788904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0660a-3af0-4bb1-9f61-495d8719a607 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.789182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0660a-3af0-4bb1-9f61-495d8719a607 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.789457 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0660a-3af0-4bb1-9f61-495d8719a607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.790788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 833/1611] 10.222.0.18 () {58 vars in 1059 bytes} [Mon Aug 29 20:32:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:46.791455 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655dcb8-adfe-44f8-a886-54b6deb3c114 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:46.796221 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:46.826889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:46.861225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0660a-3af0-4bb1-9f61-495d8719a607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:46.863035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0660a-3af0-4bb1-9f61-495d8719a607 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:46.873591 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 779/1612] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:32:46.874441 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0660a-3af0-4bb1-9f61-495d8719a607 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:46.875785 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-edb0660a-3af0-4bb1-9f61-495d8719a607 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:46.876843 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655dcb8-adfe-44f8-a886-54b6deb3c114 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:46.882093 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f82448-c05f-42fc-a633-934a902d8652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:46.882905 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f82448-c05f-42fc-a633-934a902d8652 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.883254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f82448-c05f-42fc-a633-934a902d8652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.883600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f82448-c05f-42fc-a633-934a902d8652 admin admin] PATH_INFO: `/v3/projects/1a8cdc6e922348f490c740427062c261/users/a5e82b4ae7f547aeb074b3b702d78ccc/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.888010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0660a-3af0-4bb1-9f61-495d8719a607 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:46.892474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edadee-4224-4727-be00-2f3b6b6cdbbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:46.893165 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edadee-4224-4727-be00-2f3b6b6cdbbf service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.893516 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edadee-4224-4727-be00-2f3b6b6cdbbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.893938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edadee-4224-4727-be00-2f3b6b6cdbbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.901256 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f82448-c05f-42fc-a633-934a902d8652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a8cdc6e922348f490c740427062c261, user_id=a5e82b4ae7f547aeb074b3b702d78ccc, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:46.904747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f82448-c05f-42fc-a633-934a902d8652 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:46.906289 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edadee-4224-4727-be00-2f3b6b6cdbbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:46.908156 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edadee-4224-4727-be00-2f3b6b6cdbbf service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:46.923449 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 780/1613] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:32:46.924128 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 834/1614] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:32:46 2022] PUT /identity/v3/projects/1a8cdc6e922348f490c740427062c261/users/a5e82b4ae7f547aeb074b3b702d78ccc/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:46.924271 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edadee-4224-4727-be00-2f3b6b6cdbbf service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:46.925416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88ec7a-7fad-4a56-90c6-71c1eac22c74 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:46.925701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88ec7a-7fad-4a56-90c6-71c1eac22c74 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:46.926547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88ec7a-7fad-4a56-90c6-71c1eac22c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:46.952156 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edadee-4224-4727-be00-2f3b6b6cdbbf service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:46.964562 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2b88ec7a-7fad-4a56-90c6-71c1eac22c74 None None] MFA Rules not processed for user `b2b25de07f144abcadceb35e2bad5ca2`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:46.967289 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2b88ec7a-7fad-4a56-90c6-71c1eac22c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:47.033212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c85191-8dce-446d-b816-2c30b5481d6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:47.033929 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c85191-8dce-446d-b816-2c30b5481d6b service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:47.034326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c85191-8dce-446d-b816-2c30b5481d6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:47.035019 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c85191-8dce-446d-b816-2c30b5481d6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:47.076766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 835/1615] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:46 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:47.076766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d640790-4f21-40e6-bf47-481fadcd36e2 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:47.076766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d640790-4f21-40e6-bf47-481fadcd36e2 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:47.076766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d640790-4f21-40e6-bf47-481fadcd36e2 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:47.077863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 836/1616] 10.222.0.18 () {54 vars in 962 bytes} [Mon Aug 29 20:32:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:32:47.083896 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d640790-4f21-40e6-bf47-481fadcd36e2 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:47.117179 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c85191-8dce-446d-b816-2c30b5481d6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:47.119408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c85191-8dce-446d-b816-2c30b5481d6b service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:47.132024 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 781/1617] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:32:47.133853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7c501-c445-489b-9351-5350d28d5903 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:47.134035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7c501-c445-489b-9351-5350d28d5903 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:47.134480 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7c501-c445-489b-9351-5350d28d5903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:47.178848 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d640790-4f21-40e6-bf47-481fadcd36e2 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:47.187349 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e1caa-35f6-481f-a8c8-3e4c73699259 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:47.188235 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e1caa-35f6-481f-a8c8-3e4c73699259 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:47.188594 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e1caa-35f6-481f-a8c8-3e4c73699259 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:47.189018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e1caa-35f6-481f-a8c8-3e4c73699259 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:47.189365 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-72e7c501-c445-489b-9351-5350d28d5903 None None] MFA Rules not processed for user `a5e82b4ae7f547aeb074b3b702d78ccc`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:47.191999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-72e7c501-c445-489b-9351-5350d28d5903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:47.265614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:47.301348 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e1caa-35f6-481f-a8c8-3e4c73699259 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:47.303853 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e1caa-35f6-481f-a8c8-3e4c73699259 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:47.308268 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:47.322123 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 837/1618] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:32:47.339791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e1caa-35f6-481f-a8c8-3e4c73699259 service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:47.382226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e1caa-35f6-481f-a8c8-3e4c73699259 service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:47.401517 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c3669-e275-492b-b383-ae6e59ec78e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:47.403338 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c3669-e275-492b-b383-ae6e59ec78e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:47.405846 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c3669-e275-492b-b383-ae6e59ec78e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:47.406151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c3669-e275-492b-b383-ae6e59ec78e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:47.407723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c3669-e275-492b-b383-ae6e59ec78e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:47.412821 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 782/1619] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:47.412821 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b400ecf-e067-459b-9a4d-076510aaa949 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:47.413096 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c3669-e275-492b-b383-ae6e59ec78e6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:47.413176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b400ecf-e067-459b-9a4d-076510aaa949 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:47.413176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b400ecf-e067-459b-9a4d-076510aaa949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:47.447580 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1b400ecf-e067-459b-9a4d-076510aaa949 None None] MFA Rules not processed for user `c0c44754a1c04a2b87081aba6cb509d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:47.449378 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1b400ecf-e067-459b-9a4d-076510aaa949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:47.467598 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 783/1620] 10.222.0.18 () {58 vars in 1059 bytes} [Mon Aug 29 20:32:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:47.469666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b400ecf-e067-459b-9a4d-076510aaa949 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:47.487243 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 838/1621] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:32:47 2022] POST /identity/v3/projects => generated 405 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:47.487243 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c3669-e275-492b-b383-ae6e59ec78e6 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:47.491632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b400ecf-e067-459b-9a4d-076510aaa949 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:47.634108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c3669-e275-492b-b383-ae6e59ec78e6 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:47.641563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db3d04-1d0b-48c0-aea0-78c1e40dbd2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:47.642486 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db3d04-1d0b-48c0-aea0-78c1e40dbd2f service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:47.642871 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db3d04-1d0b-48c0-aea0-78c1e40dbd2f service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:47.643283 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db3d04-1d0b-48c0-aea0-78c1e40dbd2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:47.650517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baafe48-49ee-4586-9ef2-34c0b9cae85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:47.651188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baafe48-49ee-4586-9ef2-34c0b9cae85c admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:47.652233 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baafe48-49ee-4586-9ef2-34c0b9cae85c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:47.652775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baafe48-49ee-4586-9ef2-34c0b9cae85c admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:47.653618 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baafe48-49ee-4586-9ef2-34c0b9cae85c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:47.658933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baafe48-49ee-4586-9ef2-34c0b9cae85c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:47.727620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 784/1622] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:32:47 2022] POST /identity/v3/projects => generated 377 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:47.728623 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baafe48-49ee-4586-9ef2-34c0b9cae85c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:47.752041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db3d04-1d0b-48c0-aea0-78c1e40dbd2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:47.753612 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db3d04-1d0b-48c0-aea0-78c1e40dbd2f service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:47.777974 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 839/1623] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:32:47.779080 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db3d04-1d0b-48c0-aea0-78c1e40dbd2f service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:47.803516 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db3d04-1d0b-48c0-aea0-78c1e40dbd2f service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:47.843383 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baafe48-49ee-4586-9ef2-34c0b9cae85c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:47.849139 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ded11b-3125-45fb-b7db-7e3bafd64407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:47.850030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ded11b-3125-45fb-b7db-7e3bafd64407 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:47.850377 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ded11b-3125-45fb-b7db-7e3bafd64407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:47.850871 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ded11b-3125-45fb-b7db-7e3bafd64407 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:47.852041 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ded11b-3125-45fb-b7db-7e3bafd64407 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:47.854079 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ded11b-3125-45fb-b7db-7e3bafd64407 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:47.896148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 785/1624] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:32:47 2022] POST /identity/v3/users => generated 309 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:47.900310 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ded11b-3125-45fb-b7db-7e3bafd64407 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:47.917480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:47.922535 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ded11b-3125-45fb-b7db-7e3bafd64407 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:47.933047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7658b12-a4ea-44d6-9043-a3f3c3a65d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:47.933780 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7658b12-a4ea-44d6-9043-a3f3c3a65d48 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:47.934059 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7658b12-a4ea-44d6-9043-a3f3c3a65d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:47.934329 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7658b12-a4ea-44d6-9043-a3f3c3a65d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:47.935583 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7658b12-a4ea-44d6-9043-a3f3c3a65d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:47.936577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82f3e5-772b-4233-a850-8bb81b502367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:47.937353 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82f3e5-772b-4233-a850-8bb81b502367 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:47.937844 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82f3e5-772b-4233-a850-8bb81b502367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:47.938201 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82f3e5-772b-4233-a850-8bb81b502367 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:47.938942 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82f3e5-772b-4233-a850-8bb81b502367 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:47.939834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7658b12-a4ea-44d6-9043-a3f3c3a65d48 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:47.945752 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82f3e5-772b-4233-a850-8bb81b502367 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:47.982217 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 786/1625] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:47.992576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 840/1626] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:32:47 2022] POST /identity/v3/users => generated 295 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:47.995538 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7658b12-a4ea-44d6-9043-a3f3c3a65d48 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:47.996203 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858caf5-ce97-419b-a230-44e3fa99f862 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:47.996628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858caf5-ce97-419b-a230-44e3fa99f862 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:47.997017 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858caf5-ce97-419b-a230-44e3fa99f862 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:47.999341 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 841/1627] 10.222.0.18 () {54 vars in 912 bytes} [Mon Aug 29 20:32:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:32:48.002675 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858caf5-ce97-419b-a230-44e3fa99f862 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:48.026550 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7658b12-a4ea-44d6-9043-a3f3c3a65d48 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:48.027384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858caf5-ce97-419b-a230-44e3fa99f862 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:48.040403 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0472c-9b38-4187-b231-6bbf74904f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:48.041487 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0472c-9b38-4187-b231-6bbf74904f0b admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:48.042650 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0472c-9b38-4187-b231-6bbf74904f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:48.043016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4115a-8dce-47c6-9772-f67895a92d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:48.043504 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0472c-9b38-4187-b231-6bbf74904f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:48.043702 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4115a-8dce-47c6-9772-f67895a92d05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:48.044029 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4115a-8dce-47c6-9772-f67895a92d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:48.045429 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0472c-9b38-4187-b231-6bbf74904f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:48.046025 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4115a-8dce-47c6-9772-f67895a92d05 admin admin] PATH_INFO: `/v3/projects/6367cafb1e524b33bb507ca872a8b854/users/2c8de060df7b4dfeae360e94601356d9/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:48.052430 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0472c-9b38-4187-b231-6bbf74904f0b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:48.073420 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 842/1628] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:48.078140 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4115a-8dce-47c6-9772-f67895a92d05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6367cafb1e524b33bb507ca872a8b854, user_id=2c8de060df7b4dfeae360e94601356d9, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:48.079950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f14ff-4677-4d8f-9720-5b435d325b0d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:48.080900 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4115a-8dce-47c6-9772-f67895a92d05 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:48.081883 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f14ff-4677-4d8f-9720-5b435d325b0d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:48.083971 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f14ff-4677-4d8f-9720-5b435d325b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:48.112247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 787/1629] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:32:47 2022] PUT /identity/v3/projects/6367cafb1e524b33bb507ca872a8b854/users/2c8de060df7b4dfeae360e94601356d9/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:48.112247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4115a-8dce-47c6-9772-f67895a92d05 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:48.118456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:48.136659 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2b7f14ff-4677-4d8f-9720-5b435d325b0d None None] MFA Rules not processed for user `c0c44754a1c04a2b87081aba6cb509d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:48.138445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2b7f14ff-4677-4d8f-9720-5b435d325b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:48.254558 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4115a-8dce-47c6-9772-f67895a92d05 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:48.262044 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b718a95-075f-4549-8ea4-d18b15847637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:48.263034 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b718a95-075f-4549-8ea4-d18b15847637 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:48.263414 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b718a95-075f-4549-8ea4-d18b15847637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:48.264137 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b718a95-075f-4549-8ea4-d18b15847637 admin admin] PATH_INFO: `/v3/projects/3464fe4aaa7543019c06ef1d8edb1e61/users/fba92dfe70094c9c8a9b94bcd2cd7309/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:48.304528 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b718a95-075f-4549-8ea4-d18b15847637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3464fe4aaa7543019c06ef1d8edb1e61, user_id=fba92dfe70094c9c8a9b94bcd2cd7309, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:48.304528 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b718a95-075f-4549-8ea4-d18b15847637 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:48.337986 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 788/1630] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:32:48 2022] PUT /identity/v3/projects/3464fe4aaa7543019c06ef1d8edb1e61/users/fba92dfe70094c9c8a9b94bcd2cd7309/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:48.339057 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b718a95-075f-4549-8ea4-d18b15847637 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:48.345793 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 843/1631] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 29 20:32:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:48.347092 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846c96a-91c0-4671-8e00-76a1b085ae92 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:48.347386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846c96a-91c0-4671-8e00-76a1b085ae92 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:48.347660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846c96a-91c0-4671-8e00-76a1b085ae92 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:48.349723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 844/1632] 10.222.0.18 () {54 vars in 912 bytes} [Mon Aug 29 20:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:32:48.351056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846c96a-91c0-4671-8e00-76a1b085ae92 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:48.457035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b718a95-075f-4549-8ea4-d18b15847637 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:48.463101 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846c96a-91c0-4671-8e00-76a1b085ae92 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:48.463545 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7f05a-2f27-405c-a7f2-44d31aa4e7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:48.464857 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7f05a-2f27-405c-a7f2-44d31aa4e7d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:48.465206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7f05a-2f27-405c-a7f2-44d31aa4e7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:48.465542 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7f05a-2f27-405c-a7f2-44d31aa4e7d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:48.466983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7f05a-2f27-405c-a7f2-44d31aa4e7d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:48.468637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8842b3-db15-4bc6-a724-61980fc18a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:48.468778 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7f05a-2f27-405c-a7f2-44d31aa4e7d3 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:48.469267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8842b3-db15-4bc6-a724-61980fc18a5c admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:48.469578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8842b3-db15-4bc6-a724-61980fc18a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:48.469883 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8842b3-db15-4bc6-a724-61980fc18a5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:48.470579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8842b3-db15-4bc6-a724-61980fc18a5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:48.472276 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8842b3-db15-4bc6-a724-61980fc18a5c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:48.484935 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 845/1633] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:48.486277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b39b1e-e78d-427b-8483-90876c80bc4a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:48.486563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b39b1e-e78d-427b-8483-90876c80bc4a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:48.486839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b39b1e-e78d-427b-8483-90876c80bc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:48.491520 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 789/1634] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:48.492682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7f05a-2f27-405c-a7f2-44d31aa4e7d3 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:48.511373 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-89b39b1e-e78d-427b-8483-90876c80bc4a None None] MFA Rules not processed for user `c0c44754a1c04a2b87081aba6cb509d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:48.512768 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-89b39b1e-e78d-427b-8483-90876c80bc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:48.601023 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:48.602723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7f05a-2f27-405c-a7f2-44d31aa4e7d3 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:48.613949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606652a9-cd9d-42d5-bd7f-efd7083e9edd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:48.614959 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606652a9-cd9d-42d5-bd7f-efd7083e9edd service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:48.614959 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606652a9-cd9d-42d5-bd7f-efd7083e9edd service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:48.615539 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606652a9-cd9d-42d5-bd7f-efd7083e9edd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:48.636776 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 846/1635] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 29 20:32:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:48.637644 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b39b1e-e78d-427b-8483-90876c80bc4a None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:48.692251 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b39b1e-e78d-427b-8483-90876c80bc4a None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:48.706490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b7271-239e-4e6f-aa50-de5dcf5f61bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:48.707203 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b7271-239e-4e6f-aa50-de5dcf5f61bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:48.707477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b7271-239e-4e6f-aa50-de5dcf5f61bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:48.707754 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b7271-239e-4e6f-aa50-de5dcf5f61bf admin admin] PATH_INFO: `/v3/projects/3464fe4aaa7543019c06ef1d8edb1e61/users/fba92dfe70094c9c8a9b94bcd2cd7309/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:48.710512 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b7271-239e-4e6f-aa50-de5dcf5f61bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3464fe4aaa7543019c06ef1d8edb1e61, user_id=fba92dfe70094c9c8a9b94bcd2cd7309, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:48.713805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b7271-239e-4e6f-aa50-de5dcf5f61bf admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:48.715056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:48.756447 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 847/1636] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:32:48 2022] PUT /identity/v3/projects/3464fe4aaa7543019c06ef1d8edb1e61/users/fba92dfe70094c9c8a9b94bcd2cd7309/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:48.759551 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b7271-239e-4e6f-aa50-de5dcf5f61bf admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:48.794152 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606652a9-cd9d-42d5-bd7f-efd7083e9edd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:48.800847 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606652a9-cd9d-42d5-bd7f-efd7083e9edd service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:48.842180 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 790/1637] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:32:48.843211 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606652a9-cd9d-42d5-bd7f-efd7083e9edd service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:48.849627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b7271-239e-4e6f-aa50-de5dcf5f61bf admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:48.855382 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfadde6-872b-4a33-bca0-59bb115ea337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:48.857289 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfadde6-872b-4a33-bca0-59bb115ea337 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:48.857741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfadde6-872b-4a33-bca0-59bb115ea337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:48.858118 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfadde6-872b-4a33-bca0-59bb115ea337 admin admin] PATH_INFO: `/v3/projects/6367cafb1e524b33bb507ca872a8b854/users/2c8de060df7b4dfeae360e94601356d9/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:48.860774 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfadde6-872b-4a33-bca0-59bb115ea337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6367cafb1e524b33bb507ca872a8b854, user_id=2c8de060df7b4dfeae360e94601356d9, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:48.863826 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfadde6-872b-4a33-bca0-59bb115ea337 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:48.888117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 848/1638] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:32:48 2022] PUT /identity/v3/projects/6367cafb1e524b33bb507ca872a8b854/users/2c8de060df7b4dfeae360e94601356d9/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:48.888355 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606652a9-cd9d-42d5-bd7f-efd7083e9edd service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:48.888464 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a775dd-3754-4086-9929-dbe7ab14b25b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:48.888464 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a775dd-3754-4086-9929-dbe7ab14b25b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:48.888464 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a775dd-3754-4086-9929-dbe7ab14b25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:48.994405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-63a775dd-3754-4086-9929-dbe7ab14b25b None None] MFA Rules not processed for user `fba92dfe70094c9c8a9b94bcd2cd7309`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:49.000651 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-63a775dd-3754-4086-9929-dbe7ab14b25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:49.047100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859ed16-be95-4fbe-b76d-9dd177160236 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:49.047747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859ed16-be95-4fbe-b76d-9dd177160236 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:49.048109 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859ed16-be95-4fbe-b76d-9dd177160236 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:49.048519 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8859ed16-be95-4fbe-b76d-9dd177160236 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:49.201574 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 849/1639] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:49.203244 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c31a9-646f-48e2-a2ee-fd1bdb615744 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:49.203628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c31a9-646f-48e2-a2ee-fd1bdb615744 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:49.204027 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c31a9-646f-48e2-a2ee-fd1bdb615744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:49.224760 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859ed16-be95-4fbe-b76d-9dd177160236 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:49.228535 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8859ed16-be95-4fbe-b76d-9dd177160236 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:49.264383 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 791/1640] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:32:49.272956 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a83c31a9-646f-48e2-a2ee-fd1bdb615744 None None] MFA Rules not processed for user `2c8de060df7b4dfeae360e94601356d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:49.274582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a83c31a9-646f-48e2-a2ee-fd1bdb615744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:49.383683 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859ed16-be95-4fbe-b76d-9dd177160236 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:49.398031 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8859ed16-be95-4fbe-b76d-9dd177160236 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:49.406021 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6476b4-a9f7-42b3-83c0-1828e70f1e1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:49.406660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6476b4-a9f7-42b3-83c0-1828e70f1e1c service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:49.406931 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6476b4-a9f7-42b3-83c0-1828e70f1e1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:49.407206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6476b4-a9f7-42b3-83c0-1828e70f1e1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:49.444571 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:49.507550 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 850/1641] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:49 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:49.533123 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6476b4-a9f7-42b3-83c0-1828e70f1e1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:49.536155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6476b4-a9f7-42b3-83c0-1828e70f1e1c service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:49.556488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 792/1642] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:32:50.110264 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c31a9-646f-48e2-a2ee-fd1bdb615744 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:50.138842 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c31a9-646f-48e2-a2ee-fd1bdb615744 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:50.151263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc9971-b335-40e8-beb9-c7cd8ab4996c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:50.151908 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc9971-b335-40e8-beb9-c7cd8ab4996c service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:50.152245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc9971-b335-40e8-beb9-c7cd8ab4996c service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:50.152521 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc9971-b335-40e8-beb9-c7cd8ab4996c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:50.192831 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc9971-b335-40e8-beb9-c7cd8ab4996c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:50.194305 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc9971-b335-40e8-beb9-c7cd8ab4996c service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:50.217603 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 851/1643] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:50 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:32:50.562420 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6476b4-a9f7-42b3-83c0-1828e70f1e1c service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:50.570141 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:50.714948 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6476b4-a9f7-42b3-83c0-1828e70f1e1c service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:50.730132 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0794981-27ea-4b77-b2da-5546e85504a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:50.737380 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0794981-27ea-4b77-b2da-5546e85504a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:50.737760 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0794981-27ea-4b77-b2da-5546e85504a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:50.738021 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0794981-27ea-4b77-b2da-5546e85504a6 admin admin] PATH_INFO: `/v3/users/abefeff1898c4f27b14f11fb8c6ab740` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:50.739461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0794981-27ea-4b77-b2da-5546e85504a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abefeff1898c4f27b14f11fb8c6ab740)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:50.741496 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0794981-27ea-4b77-b2da-5546e85504a6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:50.850191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc9971-b335-40e8-beb9-c7cd8ab4996c service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:50.869146 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d0794981-27ea-4b77-b2da-5546e85504a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abefeff1898c4f27b14f11fb8c6ab740'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:50.869605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc9971-b335-40e8-beb9-c7cd8ab4996c service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:50.876636 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad168db-08b2-440e-bdaa-c01f41b69bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:50.877807 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad168db-08b2-440e-bdaa-c01f41b69bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:50.878310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad168db-08b2-440e-bdaa-c01f41b69bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:50.912288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad168db-08b2-440e-bdaa-c01f41b69bb5 admin admin] PATH_INFO: `/v3/users/b2b25de07f144abcadceb35e2bad5ca2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:50.912288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad168db-08b2-440e-bdaa-c01f41b69bb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2b25de07f144abcadceb35e2bad5ca2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:50.912288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: WARNING py.warnings [None req-7ad168db-08b2-440e-bdaa-c01f41b69bb5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:32:50.912288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: warnings.warn(msg) Aug 29 20:32:50.912288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: Aug 29 20:32:50.917270 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d0794981-27ea-4b77-b2da-5546e85504a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abefeff1898c4f27b14f11fb8c6ab740'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:50.917381 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad168db-08b2-440e-bdaa-c01f41b69bb5 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:50.917853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d0794981-27ea-4b77-b2da-5546e85504a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abefeff1898c4f27b14f11fb8c6ab740'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:50.918668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d0794981-27ea-4b77-b2da-5546e85504a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abefeff1898c4f27b14f11fb8c6ab740'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:50.967405 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d0794981-27ea-4b77-b2da-5546e85504a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abefeff1898c4f27b14f11fb8c6ab740'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:50.993353 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 793/1644] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:32:50 2022] DELETE /identity/v3/users/abefeff1898c4f27b14f11fb8c6ab740 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:50.996979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e7437-3314-47e3-bca6-cd1f5e310f1b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:50.997303 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e7437-3314-47e3-bca6-cd1f5e310f1b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:50.997520 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e7437-3314-47e3-bca6-cd1f5e310f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:51.034815 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-bb7e7437-3314-47e3-bca6-cd1f5e310f1b None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:51.043548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bb7e7437-3314-47e3-bca6-cd1f5e310f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:51.076415 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7ad168db-08b2-440e-bdaa-c01f41b69bb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2b25de07f144abcadceb35e2bad5ca2'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:51.112535 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7ad168db-08b2-440e-bdaa-c01f41b69bb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2b25de07f144abcadceb35e2bad5ca2'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:51.136003 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7ad168db-08b2-440e-bdaa-c01f41b69bb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2b25de07f144abcadceb35e2bad5ca2'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:51.136855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7ad168db-08b2-440e-bdaa-c01f41b69bb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2b25de07f144abcadceb35e2bad5ca2'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:51.137498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7ad168db-08b2-440e-bdaa-c01f41b69bb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2b25de07f144abcadceb35e2bad5ca2'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:51.177942 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 852/1645] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:32:50 2022] DELETE /identity/v3/users/b2b25de07f144abcadceb35e2bad5ca2 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:32:51.180165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d53a84-ff2e-4d28-882c-ef788b12585f None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:51.180744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d53a84-ff2e-4d28-882c-ef788b12585f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:51.181173 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d53a84-ff2e-4d28-882c-ef788b12585f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:51.204507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-25d53a84-ff2e-4d28-882c-ef788b12585f None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:51.206210 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-25d53a84-ff2e-4d28-882c-ef788b12585f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:51.250970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 794/1646] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:51.282415 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e7437-3314-47e3-bca6-cd1f5e310f1b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:51.284175 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bb7e7437-3314-47e3-bca6-cd1f5e310f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:51.382192 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:51.489254 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e7437-3314-47e3-bca6-cd1f5e310f1b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:51.555423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 853/1647] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:51.616764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d53a84-ff2e-4d28-882c-ef788b12585f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:51.619722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-25d53a84-ff2e-4d28-882c-ef788b12585f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:51.635434 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b13b8-321f-4212-9817-f1225a3eee3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:51.636174 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b13b8-321f-4212-9817-f1225a3eee3e service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:51.636369 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b13b8-321f-4212-9817-f1225a3eee3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:51.636739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b13b8-321f-4212-9817-f1225a3eee3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:51.650096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b13b8-321f-4212-9817-f1225a3eee3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:51.651781 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b13b8-321f-4212-9817-f1225a3eee3e service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:51.668061 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 795/1648] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:32:51.738292 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:51.769746 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d53a84-ff2e-4d28-882c-ef788b12585f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:51.780996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad2b03-6cda-4a1a-b69d-016c2493984a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:51.781710 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad2b03-6cda-4a1a-b69d-016c2493984a service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:51.781995 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad2b03-6cda-4a1a-b69d-016c2493984a service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:51.782272 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad2b03-6cda-4a1a-b69d-016c2493984a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:51.814217 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ad2b03-6cda-4a1a-b69d-016c2493984a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:51.816100 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ad2b03-6cda-4a1a-b69d-016c2493984a service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:51.838541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 854/1649] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:32:53.359426 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b13b8-321f-4212-9817-f1225a3eee3e service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:53.361304 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a56b13b8-321f-4212-9817-f1225a3eee3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:53.367287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad2b03-6cda-4a1a-b69d-016c2493984a service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:53.369378 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-10ad2b03-6cda-4a1a-b69d-016c2493984a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:53.394293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad2b03-6cda-4a1a-b69d-016c2493984a service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:53.403279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac921624-a39d-4087-934a-3c844873ad5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:53.404676 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac921624-a39d-4087-934a-3c844873ad5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:53.405659 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac921624-a39d-4087-934a-3c844873ad5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:53.406068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac921624-a39d-4087-934a-3c844873ad5b admin admin] PATH_INFO: `/v3/projects/744393782ba8406c9c1c9f5819696be0` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:53.407853 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac921624-a39d-4087-934a-3c844873ad5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=744393782ba8406c9c1c9f5819696be0)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:53.410443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac921624-a39d-4087-934a-3c844873ad5b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:53.427089 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b13b8-321f-4212-9817-f1225a3eee3e service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:53.447339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a9d69-cc29-428b-bd76-035c69ed7bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:53.449084 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a9d69-cc29-428b-bd76-035c69ed7bb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:53.453308 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a9d69-cc29-428b-bd76-035c69ed7bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:53.454929 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a9d69-cc29-428b-bd76-035c69ed7bb9 admin admin] PATH_INFO: `/v3/projects/e97ceb2f66cf4762b62ffbedcf11b9a1` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:53.458036 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a9d69-cc29-428b-bd76-035c69ed7bb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e97ceb2f66cf4762b62ffbedcf11b9a1)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:53.462775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: WARNING py.warnings [None req-bb4a9d69-cc29-428b-bd76-035c69ed7bb9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 29 20:32:53.462775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: warnings.warn(msg) Aug 29 20:32:53.462775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: Aug 29 20:32:53.465012 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a9d69-cc29-428b-bd76-035c69ed7bb9 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:53.557768 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ac921624-a39d-4087-934a-3c844873ad5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '744393782ba8406c9c1c9f5819696be0', 'request_id': 'req-ac921624-a39d-4087-934a-3c844873ad5b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:53.559244 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-bb4a9d69-cc29-428b-bd76-035c69ed7bb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e97ceb2f66cf4762b62ffbedcf11b9a1', 'request_id': 'req-bb4a9d69-cc29-428b-bd76-035c69ed7bb9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:53.568119 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-bb4a9d69-cc29-428b-bd76-035c69ed7bb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e97ceb2f66cf4762b62ffbedcf11b9a1', 'request_id': 'req-bb4a9d69-cc29-428b-bd76-035c69ed7bb9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:53.588017 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-bb4a9d69-cc29-428b-bd76-035c69ed7bb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e97ceb2f66cf4762b62ffbedcf11b9a1', 'request_id': 'req-bb4a9d69-cc29-428b-bd76-035c69ed7bb9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:53.590426 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ac921624-a39d-4087-934a-3c844873ad5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '744393782ba8406c9c1c9f5819696be0', 'request_id': 'req-ac921624-a39d-4087-934a-3c844873ad5b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:53.610394 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ac921624-a39d-4087-934a-3c844873ad5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '744393782ba8406c9c1c9f5819696be0', 'request_id': 'req-ac921624-a39d-4087-934a-3c844873ad5b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:53.625550 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ac921624-a39d-4087-934a-3c844873ad5b admin admin] The token cache is being invalidate because project 744393782ba8406c9c1c9f5819696be0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:32:53.625550 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ac921624-a39d-4087-934a-3c844873ad5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:53.625905 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ac921624-a39d-4087-934a-3c844873ad5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:53.628829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 855/1650] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:32:53 2022] DELETE /identity/v3/projects/744393782ba8406c9c1c9f5819696be0 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:53.633229 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-bb4a9d69-cc29-428b-bd76-035c69ed7bb9 admin admin] The token cache is being invalidate because project e97ceb2f66cf4762b62ffbedcf11b9a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:32:53.633357 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac921624-a39d-4087-934a-3c844873ad5b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:53.633829 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-bb4a9d69-cc29-428b-bd76-035c69ed7bb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:53.634660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-bb4a9d69-cc29-428b-bd76-035c69ed7bb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:53.634810 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ac921624-a39d-4087-934a-3c844873ad5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:53.641996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 796/1651] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:32:53 2022] DELETE /identity/v3/projects/e97ceb2f66cf4762b62ffbedcf11b9a1 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:53.643269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62817c3-b97d-444d-bb4e-2e8dde604fca None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:53.643651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62817c3-b97d-444d-bb4e-2e8dde604fca None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:53.644080 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62817c3-b97d-444d-bb4e-2e8dde604fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:53.713459 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e62817c3-b97d-444d-bb4e-2e8dde604fca None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:53.715541 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e62817c3-b97d-444d-bb4e-2e8dde604fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:53.733108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac921624-a39d-4087-934a-3c844873ad5b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:53.851058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc150653-8464-4340-a2b1-20c27b485984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:53.851718 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc150653-8464-4340-a2b1-20c27b485984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:53.852034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc150653-8464-4340-a2b1-20c27b485984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:53.852310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc150653-8464-4340-a2b1-20c27b485984 admin admin] PATH_INFO: `/v3/users/2c8de060df7b4dfeae360e94601356d9` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:53.854038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc150653-8464-4340-a2b1-20c27b485984 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c8de060df7b4dfeae360e94601356d9)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:53.856396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc150653-8464-4340-a2b1-20c27b485984 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:53.878931 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:53.908039 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 797/1652] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:53.913666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccfca7-4470-450e-992a-bda4c84e9235 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:53.914476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccfca7-4470-450e-992a-bda4c84e9235 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:53.914476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ccfca7-4470-450e-992a-bda4c84e9235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:53.949313 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:53.956840 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-dc150653-8464-4340-a2b1-20c27b485984 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c8de060df7b4dfeae360e94601356d9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:53.966839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-dc150653-8464-4340-a2b1-20c27b485984 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c8de060df7b4dfeae360e94601356d9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:53.980615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-dc150653-8464-4340-a2b1-20c27b485984 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c8de060df7b4dfeae360e94601356d9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:53.981627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-dc150653-8464-4340-a2b1-20c27b485984 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c8de060df7b4dfeae360e94601356d9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:53.983293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-dc150653-8464-4340-a2b1-20c27b485984 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c8de060df7b4dfeae360e94601356d9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:54.019547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 856/1653] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:32:53 2022] DELETE /identity/v3/users/2c8de060df7b4dfeae360e94601356d9 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:54.020457 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d8ccfca7-4470-450e-992a-bda4c84e9235 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:54.021699 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d8ccfca7-4470-450e-992a-bda4c84e9235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:54.182845 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 798/1654] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:54.188883 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc150653-8464-4340-a2b1-20c27b485984 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:54.206296 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc150653-8464-4340-a2b1-20c27b485984 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:54.220187 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b462d-97d4-47b0-a543-5fb964912cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:54.221172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b462d-97d4-47b0-a543-5fb964912cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:54.221926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b462d-97d4-47b0-a543-5fb964912cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:54.222369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b462d-97d4-47b0-a543-5fb964912cd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:54.227615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b462d-97d4-47b0-a543-5fb964912cd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:32:54.228142 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b462d-97d4-47b0-a543-5fb964912cd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:54.230182 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b462d-97d4-47b0-a543-5fb964912cd9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:54.251663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccfca7-4470-450e-992a-bda4c84e9235 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:54.263238 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 857/1655] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:32:54.264618 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ccfca7-4470-450e-992a-bda4c84e9235 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:54.275818 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5986b-36f7-4872-ae64-ebd79dedebad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:54.276673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5986b-36f7-4872-ae64-ebd79dedebad admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:54.277055 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5986b-36f7-4872-ae64-ebd79dedebad admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:54.277433 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a5986b-36f7-4872-ae64-ebd79dedebad admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:54.278217 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5986b-36f7-4872-ae64-ebd79dedebad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:54.281841 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a5986b-36f7-4872-ae64-ebd79dedebad admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:54.370457 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 799/1656] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:32:54 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:54.372117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b462d-97d4-47b0-a543-5fb964912cd9 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:54.397016 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b462d-97d4-47b0-a543-5fb964912cd9 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:54.561898 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e51c55-edb8-4847-bc36-d3110cb42424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:54.563231 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e51c55-edb8-4847-bc36-d3110cb42424 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:54.563518 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e51c55-edb8-4847-bc36-d3110cb42424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:54.563795 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e51c55-edb8-4847-bc36-d3110cb42424 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:54.564938 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e51c55-edb8-4847-bc36-d3110cb42424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:54.567792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e51c55-edb8-4847-bc36-d3110cb42424 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:54.616766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 858/1657] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:32:54 2022] POST /identity/v3/users => generated 291 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:54.620835 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5986b-36f7-4872-ae64-ebd79dedebad admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:54.646225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a5986b-36f7-4872-ae64-ebd79dedebad admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:54.661993 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d2ddb-b991-4ce6-828e-fbe09edf50fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:54.662675 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d2ddb-b991-4ce6-828e-fbe09edf50fb admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:54.662958 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d2ddb-b991-4ce6-828e-fbe09edf50fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:54.666582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d2ddb-b991-4ce6-828e-fbe09edf50fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:54.668407 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d2ddb-b991-4ce6-828e-fbe09edf50fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:54.670051 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d2ddb-b991-4ce6-828e-fbe09edf50fb admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:54.704509 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 800/1658] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:54.708834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e51c55-edb8-4847-bc36-d3110cb42424 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:54.724553 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e51c55-edb8-4847-bc36-d3110cb42424 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:54.743802 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f5038-aacf-495f-b116-52e09d0c3cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:54.746356 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f5038-aacf-495f-b116-52e09d0c3cac admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:54.747320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f5038-aacf-495f-b116-52e09d0c3cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:54.747636 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f5038-aacf-495f-b116-52e09d0c3cac admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:54.750469 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f5038-aacf-495f-b116-52e09d0c3cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:54.758111 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f5038-aacf-495f-b116-52e09d0c3cac admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:54.800311 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 859/1659] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:54.803772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d2ddb-b991-4ce6-828e-fbe09edf50fb admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:54.820131 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d2ddb-b991-4ce6-828e-fbe09edf50fb admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:54.830223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf4ad7-e878-4edf-80f7-b83a3f7f66ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:54.830926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf4ad7-e878-4edf-80f7-b83a3f7f66ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:54.831209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf4ad7-e878-4edf-80f7-b83a3f7f66ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:54.831487 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf4ad7-e878-4edf-80f7-b83a3f7f66ae admin admin] PATH_INFO: `/v3/projects/fcb16ab0cf4a4428ade065b9f7468a5e/users/96794c9e9dee4a1ca6459b397fdcea58/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:54.870732 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf4ad7-e878-4edf-80f7-b83a3f7f66ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcb16ab0cf4a4428ade065b9f7468a5e, user_id=96794c9e9dee4a1ca6459b397fdcea58, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:54.874737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf4ad7-e878-4edf-80f7-b83a3f7f66ae admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:54.915766 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 801/1660] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:32:54 2022] PUT /identity/v3/projects/fcb16ab0cf4a4428ade065b9f7468a5e/users/96794c9e9dee4a1ca6459b397fdcea58/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:54.917775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b7384-ad4e-4dc5-a3a0-42bc7aad36f0 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:54.918093 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b7384-ad4e-4dc5-a3a0-42bc7aad36f0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:54.918372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b7384-ad4e-4dc5-a3a0-42bc7aad36f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:54.990694 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-7d8b7384-ad4e-4dc5-a3a0-42bc7aad36f0 None None] MFA Rules not processed for user `96794c9e9dee4a1ca6459b397fdcea58`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:54.993005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7d8b7384-ad4e-4dc5-a3a0-42bc7aad36f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:55.114690 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:55.177270 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 860/1661] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:55.179597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf4ad7-e878-4edf-80f7-b83a3f7f66ae admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:55.372691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf4ad7-e878-4edf-80f7-b83a3f7f66ae admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:55.383808 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942ee06-b85a-4dd7-98e3-e26c89b0916e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:55.385091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942ee06-b85a-4dd7-98e3-e26c89b0916e admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:55.393877 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942ee06-b85a-4dd7-98e3-e26c89b0916e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:55.394243 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942ee06-b85a-4dd7-98e3-e26c89b0916e admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:55.394949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942ee06-b85a-4dd7-98e3-e26c89b0916e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:55.397027 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942ee06-b85a-4dd7-98e3-e26c89b0916e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:55.465952 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 802/1662] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:32:55 2022] POST /identity/v3/projects => generated 379 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:55.471431 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b7384-ad4e-4dc5-a3a0-42bc7aad36f0 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:55.493207 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b7384-ad4e-4dc5-a3a0-42bc7aad36f0 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:55.609722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856d75a-b311-499b-b469-bca8f4e74303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:55.610323 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856d75a-b311-499b-b469-bca8f4e74303 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:55.610610 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856d75a-b311-499b-b469-bca8f4e74303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:55.610890 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856d75a-b311-499b-b469-bca8f4e74303 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:55.611551 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856d75a-b311-499b-b469-bca8f4e74303 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:55.614073 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856d75a-b311-499b-b469-bca8f4e74303 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:55.655997 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 861/1663] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:32:55 2022] POST /identity/v3/users => generated 296 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:55.656809 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942ee06-b85a-4dd7-98e3-e26c89b0916e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:55.671866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942ee06-b85a-4dd7-98e3-e26c89b0916e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:55.691016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c974a9-d0d5-4c48-a37d-41b1b2588820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:55.691989 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c974a9-d0d5-4c48-a37d-41b1b2588820 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:55.692442 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c974a9-d0d5-4c48-a37d-41b1b2588820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:55.692861 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c974a9-d0d5-4c48-a37d-41b1b2588820 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:55.699042 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c974a9-d0d5-4c48-a37d-41b1b2588820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:55.704282 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c974a9-d0d5-4c48-a37d-41b1b2588820 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:55.744180 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 803/1664] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:55.749158 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856d75a-b311-499b-b469-bca8f4e74303 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:55.765309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856d75a-b311-499b-b469-bca8f4e74303 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:55.798767 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457225d-f8f2-4ac7-97df-8e2b72dfa0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:55.798767 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457225d-f8f2-4ac7-97df-8e2b72dfa0c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:55.798767 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457225d-f8f2-4ac7-97df-8e2b72dfa0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:55.798767 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457225d-f8f2-4ac7-97df-8e2b72dfa0c8 admin admin] PATH_INFO: `/v3/projects/13409c26096a440aae9b98eba1ba0991/users/da370c57ce8e4c83b54527580f4f70d8/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:55.830519 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457225d-f8f2-4ac7-97df-8e2b72dfa0c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13409c26096a440aae9b98eba1ba0991, user_id=da370c57ce8e4c83b54527580f4f70d8, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:55.834266 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457225d-f8f2-4ac7-97df-8e2b72dfa0c8 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:55.853733 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 862/1665] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:32:55 2022] PUT /identity/v3/projects/13409c26096a440aae9b98eba1ba0991/users/da370c57ce8e4c83b54527580f4f70d8/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:55.857486 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c974a9-d0d5-4c48-a37d-41b1b2588820 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:55.873011 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c974a9-d0d5-4c48-a37d-41b1b2588820 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:55.906123 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:55.969882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e1ff0-5af8-4f13-aa31-9e733b2c7aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:55.971201 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e1ff0-5af8-4f13-aa31-9e733b2c7aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:55.971690 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e1ff0-5af8-4f13-aa31-9e733b2c7aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:55.972214 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e1ff0-5af8-4f13-aa31-9e733b2c7aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:55.973396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e1ff0-5af8-4f13-aa31-9e733b2c7aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:55.975471 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e1ff0-5af8-4f13-aa31-9e733b2c7aa6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:55.993406 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457225d-f8f2-4ac7-97df-8e2b72dfa0c8 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:55.993927 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 804/1666] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:56.009191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457225d-f8f2-4ac7-97df-8e2b72dfa0c8 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:56.021702 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcc5ad-ba67-472c-a81f-b46196901a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:56.023436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcc5ad-ba67-472c-a81f-b46196901a8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:56.023436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcc5ad-ba67-472c-a81f-b46196901a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:56.023672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcc5ad-ba67-472c-a81f-b46196901a8e admin admin] PATH_INFO: `/v3/projects/13409c26096a440aae9b98eba1ba0991/users/da370c57ce8e4c83b54527580f4f70d8/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:56.028807 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcc5ad-ba67-472c-a81f-b46196901a8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=13409c26096a440aae9b98eba1ba0991, user_id=da370c57ce8e4c83b54527580f4f70d8, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:56.032351 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcc5ad-ba67-472c-a81f-b46196901a8e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:56.072951 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 863/1667] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:32:55 2022] PUT /identity/v3/projects/13409c26096a440aae9b98eba1ba0991/users/da370c57ce8e4c83b54527580f4f70d8/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:32:56.075733 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243797b-d3a2-450a-b3b1-a8ee09412d90 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:56.076462 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243797b-d3a2-450a-b3b1-a8ee09412d90 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:56.076959 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2243797b-d3a2-450a-b3b1-a8ee09412d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:56.145710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-2243797b-d3a2-450a-b3b1-a8ee09412d90 None None] MFA Rules not processed for user `da370c57ce8e4c83b54527580f4f70d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:56.149269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2243797b-d3a2-450a-b3b1-a8ee09412d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:56.364451 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 805/1668] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:56 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:56.371133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcc5ad-ba67-472c-a81f-b46196901a8e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:56.372930 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-57dcc5ad-ba67-472c-a81f-b46196901a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:56.502585 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcc5ad-ba67-472c-a81f-b46196901a8e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:56.510273 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67df34-f805-4c24-b3e7-364a165ca346 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:56.516644 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67df34-f805-4c24-b3e7-364a165ca346 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:56.516934 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67df34-f805-4c24-b3e7-364a165ca346 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:56.517213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67df34-f805-4c24-b3e7-364a165ca346 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:56.540630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67df34-f805-4c24-b3e7-364a165ca346 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:56.542506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67df34-f805-4c24-b3e7-364a165ca346 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:56.575116 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 864/1669] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:56 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:32:56.706275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243797b-d3a2-450a-b3b1-a8ee09412d90 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:56.708626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2243797b-d3a2-450a-b3b1-a8ee09412d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:57.030456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2243797b-d3a2-450a-b3b1-a8ee09412d90 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:57.058507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a63701-27cd-4c7e-9fb0-d0090c56819b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:57.058979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a63701-27cd-4c7e-9fb0-d0090c56819b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:57.059265 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a63701-27cd-4c7e-9fb0-d0090c56819b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:57.059543 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a63701-27cd-4c7e-9fb0-d0090c56819b admin admin] PATH_INFO: `/v3/projects/6367cafb1e524b33bb507ca872a8b854` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:57.071404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a63701-27cd-4c7e-9fb0-d0090c56819b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6367cafb1e524b33bb507ca872a8b854)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:57.077899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a63701-27cd-4c7e-9fb0-d0090c56819b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:57.288092 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:57.290488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f3a63701-27cd-4c7e-9fb0-d0090c56819b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6367cafb1e524b33bb507ca872a8b854', 'request_id': 'req-f3a63701-27cd-4c7e-9fb0-d0090c56819b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:57.331067 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f3a63701-27cd-4c7e-9fb0-d0090c56819b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6367cafb1e524b33bb507ca872a8b854', 'request_id': 'req-f3a63701-27cd-4c7e-9fb0-d0090c56819b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:57.359396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f3a63701-27cd-4c7e-9fb0-d0090c56819b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6367cafb1e524b33bb507ca872a8b854', 'request_id': 'req-f3a63701-27cd-4c7e-9fb0-d0090c56819b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:57.407682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f3a63701-27cd-4c7e-9fb0-d0090c56819b admin admin] The token cache is being invalidate because project 6367cafb1e524b33bb507ca872a8b854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:32:57.408023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f3a63701-27cd-4c7e-9fb0-d0090c56819b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:57.414049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f3a63701-27cd-4c7e-9fb0-d0090c56819b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:57.423160 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 806/1670] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:32:56 2022] DELETE /identity/v3/projects/6367cafb1e524b33bb507ca872a8b854 => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:57.426562 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6bdc3-e554-42bf-a35c-0d243c2a9ffa None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:57.427266 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6bdc3-e554-42bf-a35c-0d243c2a9ffa None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:57.427674 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6bdc3-e554-42bf-a35c-0d243c2a9ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:57.499780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d7b6bdc3-e554-42bf-a35c-0d243c2a9ffa None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:57.506512 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d7b6bdc3-e554-42bf-a35c-0d243c2a9ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:57.573259 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a63701-27cd-4c7e-9fb0-d0090c56819b admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:57.574932 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f3a63701-27cd-4c7e-9fb0-d0090c56819b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:57.634464 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:57.691386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 865/1671] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:32:57.696777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b5f80-01c4-4ce2-a2ff-d43086219f20 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:57.698693 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b5f80-01c4-4ce2-a2ff-d43086219f20 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:57.699088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b5f80-01c4-4ce2-a2ff-d43086219f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:57.766122 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a63701-27cd-4c7e-9fb0-d0090c56819b admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:57.781599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-204b5f80-01c4-4ce2-a2ff-d43086219f20 None None] MFA Rules not processed for user `adc5854ed52042369a9b153af7da4cfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:57.782871 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dec29c-c074-4c10-af7c-fda404012d3a tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da370c57ce8e4c83b54527580f4f70d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13409c26096a440aae9b98eba1ba0991', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:57.785637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-204b5f80-01c4-4ce2-a2ff-d43086219f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:57.788585 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dec29c-c074-4c10-af7c-fda404012d3a tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:57.789133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dec29c-c074-4c10-af7c-fda404012d3a tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:57.789607 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dec29c-c074-4c10-af7c-fda404012d3a tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:57.790500 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dec29c-c074-4c10-af7c-fda404012d3a tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:32:57.791002 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dec29c-c074-4c10-af7c-fda404012d3a tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:57.793564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dec29c-c074-4c10-af7c-fda404012d3a tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:57.810996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 807/1672] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:32:57.813648 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19237c0b-073b-4abd-b929-3afd8f90f067 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:57.814186 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19237c0b-073b-4abd-b929-3afd8f90f067 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:57.814893 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19237c0b-073b-4abd-b929-3afd8f90f067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:57.864930 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-19237c0b-073b-4abd-b929-3afd8f90f067 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:57.867109 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-19237c0b-073b-4abd-b929-3afd8f90f067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:57.888303 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 808/1673] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:57.889876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19237c0b-073b-4abd-b929-3afd8f90f067 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:57.905041 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19237c0b-073b-4abd-b929-3afd8f90f067 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:57.914111 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34284432-7905-4086-a6fa-7543e747d933 tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da370c57ce8e4c83b54527580f4f70d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13409c26096a440aae9b98eba1ba0991', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:57.915122 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34284432-7905-4086-a6fa-7543e747d933 tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:57.928011 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34284432-7905-4086-a6fa-7543e747d933 tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:57.929502 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34284432-7905-4086-a6fa-7543e747d933 tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:57.932543 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34284432-7905-4086-a6fa-7543e747d933 tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:57.936850 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34284432-7905-4086-a6fa-7543e747d933 tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:57.976032 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 866/1674] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:57.977871 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b5f80-01c4-4ce2-a2ff-d43086219f20 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:58.018675 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b5f80-01c4-4ce2-a2ff-d43086219f20 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:58.028889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040fe30-9c5d-475d-9f20-d5e26d3325b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:58.029542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040fe30-9c5d-475d-9f20-d5e26d3325b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:58.029830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040fe30-9c5d-475d-9f20-d5e26d3325b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:58.030105 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040fe30-9c5d-475d-9f20-d5e26d3325b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:58.031279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040fe30-9c5d-475d-9f20-d5e26d3325b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:32:58.031588 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040fe30-9c5d-475d-9f20-d5e26d3325b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:58.036977 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040fe30-9c5d-475d-9f20-d5e26d3325b8 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:58.037487 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 809/1675] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:32:57 2022] POST /identity/v3/projects => generated 353 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:58.040417 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34284432-7905-4086-a6fa-7543e747d933 tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:58.046836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-34284432-7905-4086-a6fa-7543e747d933 tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:58.073877 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 867/1676] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:32:58.075070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040fe30-9c5d-475d-9f20-d5e26d3325b8 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:58.104996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040fe30-9c5d-475d-9f20-d5e26d3325b8 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:58.156339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34284432-7905-4086-a6fa-7543e747d933 tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:58.164575 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2347b93-1505-43a2-9b79-f117f1937369 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:58.165201 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2347b93-1505-43a2-9b79-f117f1937369 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:58.165473 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2347b93-1505-43a2-9b79-f117f1937369 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:58.165741 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2347b93-1505-43a2-9b79-f117f1937369 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:58.206121 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52383573-e141-4d8b-89cb-cc830154916e tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da370c57ce8e4c83b54527580f4f70d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13409c26096a440aae9b98eba1ba0991', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:58.207155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52383573-e141-4d8b-89cb-cc830154916e tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:58.207701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52383573-e141-4d8b-89cb-cc830154916e tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:58.208187 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52383573-e141-4d8b-89cb-cc830154916e tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:58.209100 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52383573-e141-4d8b-89cb-cc830154916e tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:32:58.209557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52383573-e141-4d8b-89cb-cc830154916e tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:58.212798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52383573-e141-4d8b-89cb-cc830154916e tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:58.229402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 868/1677] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:32:58.231013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52383573-e141-4d8b-89cb-cc830154916e tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:58.235791 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:58.307670 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2347b93-1505-43a2-9b79-f117f1937369 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:58.310381 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2347b93-1505-43a2-9b79-f117f1937369 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:58.338235 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 810/1678] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:32:58.345105 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2347b93-1505-43a2-9b79-f117f1937369 service nova] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:58.382468 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2347b93-1505-43a2-9b79-f117f1937369 service nova] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:58.395098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:58.397222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c13b23-a213-4c76-85df-1e08c1ce355f tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da370c57ce8e4c83b54527580f4f70d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13409c26096a440aae9b98eba1ba0991', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:58.398046 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c13b23-a213-4c76-85df-1e08c1ce355f tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:58.398450 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c13b23-a213-4c76-85df-1e08c1ce355f tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:58.398827 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c13b23-a213-4c76-85df-1e08c1ce355f tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] PATH_INFO: `/v3/projects/640ccbeec35546b4b605b455590b7936` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:58.400170 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c13b23-a213-4c76-85df-1e08c1ce355f tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] RBAC: Authorizing `identity:delete_project(project_id=640ccbeec35546b4b605b455590b7936)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:58.402537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c13b23-a213-4c76-85df-1e08c1ce355f tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:58.408006 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52383573-e141-4d8b-89cb-cc830154916e tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:58.425669 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c0f08-2dfc-47b7-bf87-52dec9010b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:58.428102 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c0f08-2dfc-47b7-bf87-52dec9010b97 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:58.428406 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c0f08-2dfc-47b7-bf87-52dec9010b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:58.428689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c0f08-2dfc-47b7-bf87-52dec9010b97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:58.429537 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c0f08-2dfc-47b7-bf87-52dec9010b97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:58.431626 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c0f08-2dfc-47b7-bf87-52dec9010b97 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:58.512930 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-58c13b23-a213-4c76-85df-1e08c1ce355f tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '640ccbeec35546b4b605b455590b7936', 'request_id': 'req-58c13b23-a213-4c76-85df-1e08c1ce355f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:58.533820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-58c13b23-a213-4c76-85df-1e08c1ce355f tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '640ccbeec35546b4b605b455590b7936', 'request_id': 'req-58c13b23-a213-4c76-85df-1e08c1ce355f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:58.540149 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 869/1679] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:32:58 2022] POST /identity/v3/projects => generated 369 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:58.543327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c0f08-2dfc-47b7-bf87-52dec9010b97 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:58.546309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2f0c0f08-2dfc-47b7-bf87-52dec9010b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:58.549865 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-58c13b23-a213-4c76-85df-1e08c1ce355f tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '640ccbeec35546b4b605b455590b7936', 'request_id': 'req-58c13b23-a213-4c76-85df-1e08c1ce355f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:58.575119 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-58c13b23-a213-4c76-85df-1e08c1ce355f tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] The token cache is being invalidate because project 640ccbeec35546b4b605b455590b7936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:32:58.578448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-58c13b23-a213-4c76-85df-1e08c1ce355f tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:58.579370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-58c13b23-a213-4c76-85df-1e08c1ce355f tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:58.584674 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 811/1680] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:32:58 2022] DELETE /identity/v3/projects/640ccbeec35546b4b605b455590b7936 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:58.584674 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c13b23-a213-4c76-85df-1e08c1ce355f tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:58.584674 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-58c13b23-a213-4c76-85df-1e08c1ce355f tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:58.685889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c0f08-2dfc-47b7-bf87-52dec9010b97 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:58.695577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e1db7-6e0d-46bf-9bca-2ce87b0d9631 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:58.696558 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e1db7-6e0d-46bf-9bca-2ce87b0d9631 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:58.697243 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e1db7-6e0d-46bf-9bca-2ce87b0d9631 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:58.697674 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e1db7-6e0d-46bf-9bca-2ce87b0d9631 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:58.700370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-979e1db7-6e0d-46bf-9bca-2ce87b0d9631 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:58.706470 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c13b23-a213-4c76-85df-1e08c1ce355f tempest-VolumeQuotaClassesTest-155224549 tempest-VolumeQuotaClassesTest-155224549-project-admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:58.818097 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e1db7-6e0d-46bf-9bca-2ce87b0d9631 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:58.820788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e1db7-6e0d-46bf-9bca-2ce87b0d9631 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:58.831346 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc24559-795f-49cc-8970-e5cfeb1a3765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:58.831874 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc24559-795f-49cc-8970-e5cfeb1a3765 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:58.832773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc24559-795f-49cc-8970-e5cfeb1a3765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:58.835318 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc24559-795f-49cc-8970-e5cfeb1a3765 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:58.837701 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc24559-795f-49cc-8970-e5cfeb1a3765 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:58.842051 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc24559-795f-49cc-8970-e5cfeb1a3765 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:58.855413 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 870/1681] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:32:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:32:58.910004 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:58.946607 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 812/1682] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:32:58 2022] POST /identity/v3/users => generated 285 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:32:58.951525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e1db7-6e0d-46bf-9bca-2ce87b0d9631 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:58.997355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e1db7-6e0d-46bf-9bca-2ce87b0d9631 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:59.037372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893bc0e-223d-4511-91b0-fbae8ce737b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:59.039356 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893bc0e-223d-4511-91b0-fbae8ce737b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:59.040290 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893bc0e-223d-4511-91b0-fbae8ce737b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:59.041021 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893bc0e-223d-4511-91b0-fbae8ce737b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:59.041876 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893bc0e-223d-4511-91b0-fbae8ce737b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:59.045029 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893bc0e-223d-4511-91b0-fbae8ce737b3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:59.093821 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc24559-795f-49cc-8970-e5cfeb1a3765 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:59.094158 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 871/1683] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:32:59.105583 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc24559-795f-49cc-8970-e5cfeb1a3765 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:59.140295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32367c9-cee9-422e-b9c6-b2ef9445ebaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:59.143296 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32367c9-cee9-422e-b9c6-b2ef9445ebaa admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:59.143585 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32367c9-cee9-422e-b9c6-b2ef9445ebaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:59.143869 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32367c9-cee9-422e-b9c6-b2ef9445ebaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:59.144601 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32367c9-cee9-422e-b9c6-b2ef9445ebaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:59.150628 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32367c9-cee9-422e-b9c6-b2ef9445ebaa admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:59.197842 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 813/1684] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:32:59.199212 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893bc0e-223d-4511-91b0-fbae8ce737b3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:59.225864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893bc0e-223d-4511-91b0-fbae8ce737b3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:59.241694 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2637598-69fe-4dc6-a7cd-991e8549709c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:59.242404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2637598-69fe-4dc6-a7cd-991e8549709c admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:59.242692 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2637598-69fe-4dc6-a7cd-991e8549709c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:59.247850 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2637598-69fe-4dc6-a7cd-991e8549709c admin admin] PATH_INFO: `/v3/projects/deb33d138b0f4965af6f62b28c534083/users/2d46f477264d4f2ca008278e673075e5/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:59.299255 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2637598-69fe-4dc6-a7cd-991e8549709c admin admin] RBAC: Authorizing `identity:create_grant(project_id=deb33d138b0f4965af6f62b28c534083, user_id=2d46f477264d4f2ca008278e673075e5, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:59.312317 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2637598-69fe-4dc6-a7cd-991e8549709c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:59.327095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32367c9-cee9-422e-b9c6-b2ef9445ebaa admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:59.332440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a32367c9-cee9-422e-b9c6-b2ef9445ebaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:59.358294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32367c9-cee9-422e-b9c6-b2ef9445ebaa admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:32:59.400433 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 872/1685] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:32:59 2022] PUT /identity/v3/projects/deb33d138b0f4965af6f62b28c534083/users/2d46f477264d4f2ca008278e673075e5/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:59.410450 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f815c-cbae-4ec5-acee-aeefbc3f237e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:32:59.411115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f815c-cbae-4ec5-acee-aeefbc3f237e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:59.411522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f815c-cbae-4ec5-acee-aeefbc3f237e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:59.411974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f815c-cbae-4ec5-acee-aeefbc3f237e admin admin] PATH_INFO: `/v3/users/96794c9e9dee4a1ca6459b397fdcea58` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:59.418126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc2e27-35ff-4f17-a61c-af2b18fd34e1 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:59.420121 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f815c-cbae-4ec5-acee-aeefbc3f237e admin admin] RBAC: Authorizing `identity:delete_user(user_id=96794c9e9dee4a1ca6459b397fdcea58)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:32:59.420825 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc2e27-35ff-4f17-a61c-af2b18fd34e1 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:59.421167 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc2e27-35ff-4f17-a61c-af2b18fd34e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:59.436208 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f815c-cbae-4ec5-acee-aeefbc3f237e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:32:59.528156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-abdc2e27-35ff-4f17-a61c-af2b18fd34e1 None None] MFA Rules not processed for user `2d46f477264d4f2ca008278e673075e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:59.534355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-abdc2e27-35ff-4f17-a61c-af2b18fd34e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:59.628288 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-542f815c-cbae-4ec5-acee-aeefbc3f237e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96794c9e9dee4a1ca6459b397fdcea58'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:59.657906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-542f815c-cbae-4ec5-acee-aeefbc3f237e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96794c9e9dee4a1ca6459b397fdcea58'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:59.713873 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-542f815c-cbae-4ec5-acee-aeefbc3f237e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96794c9e9dee4a1ca6459b397fdcea58'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:59.719170 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-542f815c-cbae-4ec5-acee-aeefbc3f237e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96794c9e9dee4a1ca6459b397fdcea58'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:59.768909 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:32:59.775614 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-542f815c-cbae-4ec5-acee-aeefbc3f237e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96794c9e9dee4a1ca6459b397fdcea58'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:32:59.798549 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 814/1686] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:32:59 2022] DELETE /identity/v3/users/96794c9e9dee4a1ca6459b397fdcea58 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:32:59.802750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce4849-b19d-4010-922f-1ba551ef86cc None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:32:59.803055 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce4849-b19d-4010-922f-1ba551ef86cc None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:32:59.804558 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce4849-b19d-4010-922f-1ba551ef86cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:32:59.853525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 873/1687] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:32:59.861855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc2e27-35ff-4f17-a61c-af2b18fd34e1 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:32:59.863772 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-abdc2e27-35ff-4f17-a61c-af2b18fd34e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:32:59.886591 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-25ce4849-b19d-4010-922f-1ba551ef86cc None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:32:59.893779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-25ce4849-b19d-4010-922f-1ba551ef86cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:00.180935 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc2e27-35ff-4f17-a61c-af2b18fd34e1 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:00.189719 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 815/1688] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:32:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:00.204588 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561911e-22d8-4353-98d0-f114d28a72aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:00.205667 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561911e-22d8-4353-98d0-f114d28a72aa admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:00.205861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561911e-22d8-4353-98d0-f114d28a72aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:00.206041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561911e-22d8-4353-98d0-f114d28a72aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:00.212378 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561911e-22d8-4353-98d0-f114d28a72aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:00.214614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561911e-22d8-4353-98d0-f114d28a72aa admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:00.222451 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce4849-b19d-4010-922f-1ba551ef86cc None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:00.224158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-25ce4849-b19d-4010-922f-1ba551ef86cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:00.379086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 874/1689] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:32:59 2022] POST /identity/v3/projects => generated 373 bytes in 517 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:00.385098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561911e-22d8-4353-98d0-f114d28a72aa admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:00.463506 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce4849-b19d-4010-922f-1ba551ef86cc None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:00.491645 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:00.569663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96676bee-dfe5-438f-8ae6-0aa2ef8fcdde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:00.570788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96676bee-dfe5-438f-8ae6-0aa2ef8fcdde service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:00.571324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96676bee-dfe5-438f-8ae6-0aa2ef8fcdde service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:00.571901 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96676bee-dfe5-438f-8ae6-0aa2ef8fcdde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:00.670142 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561911e-22d8-4353-98d0-f114d28a72aa admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:00.679198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96676bee-dfe5-438f-8ae6-0aa2ef8fcdde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:00.681171 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96676bee-dfe5-438f-8ae6-0aa2ef8fcdde service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:00.693986 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 816/1690] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:33:00.701443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e300d4-4053-4fa5-8f39-f9db867695fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:00.702860 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e300d4-4053-4fa5-8f39-f9db867695fa admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:00.703845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e300d4-4053-4fa5-8f39-f9db867695fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:00.704622 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e300d4-4053-4fa5-8f39-f9db867695fa admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:00.707983 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e300d4-4053-4fa5-8f39-f9db867695fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:00.710378 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e300d4-4053-4fa5-8f39-f9db867695fa admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:00.775217 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 875/1691] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:00 2022] POST /identity/v3/users => generated 293 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:00.777539 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96676bee-dfe5-438f-8ae6-0aa2ef8fcdde service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:00.800313 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96676bee-dfe5-438f-8ae6-0aa2ef8fcdde service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:00.820495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073dd46e-3fb6-4565-9638-e3e80a70efff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:00.821508 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073dd46e-3fb6-4565-9638-e3e80a70efff admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:00.822006 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073dd46e-3fb6-4565-9638-e3e80a70efff admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:00.822518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073dd46e-3fb6-4565-9638-e3e80a70efff admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:00.824590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073dd46e-3fb6-4565-9638-e3e80a70efff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:00.835341 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073dd46e-3fb6-4565-9638-e3e80a70efff admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:00.865656 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 817/1692] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:00.866328 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e300d4-4053-4fa5-8f39-f9db867695fa admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:00.880290 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e300d4-4053-4fa5-8f39-f9db867695fa admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:00.888979 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000b0b7-c0d1-499b-baf9-74d9b888aa0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:00.889860 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000b0b7-c0d1-499b-baf9-74d9b888aa0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:00.890278 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000b0b7-c0d1-499b-baf9-74d9b888aa0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:00.890679 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7000b0b7-c0d1-499b-baf9-74d9b888aa0c admin admin] PATH_INFO: `/v3/projects/fc1ccdeeb0274b89a0e7c51f317a3c9f/users/7b2277c33fa740f4ba98f44258849ad1/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:00.933687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000b0b7-c0d1-499b-baf9-74d9b888aa0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc1ccdeeb0274b89a0e7c51f317a3c9f, user_id=7b2277c33fa740f4ba98f44258849ad1, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:00.937776 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7000b0b7-c0d1-499b-baf9-74d9b888aa0c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:00.961252 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 876/1693] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:00 2022] PUT /identity/v3/projects/fc1ccdeeb0274b89a0e7c51f317a3c9f/users/7b2277c33fa740f4ba98f44258849ad1/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:00.964986 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073dd46e-3fb6-4565-9638-e3e80a70efff admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:00.988816 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073dd46e-3fb6-4565-9638-e3e80a70efff admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:01.132477 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b506b2-bc09-41db-831a-b9f14d425d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:01.133120 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b506b2-bc09-41db-831a-b9f14d425d38 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:01.133407 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b506b2-bc09-41db-831a-b9f14d425d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:01.134074 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b506b2-bc09-41db-831a-b9f14d425d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:01.134383 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b506b2-bc09-41db-831a-b9f14d425d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:01.136677 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b506b2-bc09-41db-831a-b9f14d425d38 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:01.192380 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 818/1694] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:01.196573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000b0b7-c0d1-499b-baf9-74d9b888aa0c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:01.223243 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7000b0b7-c0d1-499b-baf9-74d9b888aa0c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:01.240747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d60aa4-5c0b-4300-bfac-bf9dac6d0956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:01.243403 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d60aa4-5c0b-4300-bfac-bf9dac6d0956 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:01.244157 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d60aa4-5c0b-4300-bfac-bf9dac6d0956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:01.244635 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d60aa4-5c0b-4300-bfac-bf9dac6d0956 admin admin] PATH_INFO: `/v3/projects/fc1ccdeeb0274b89a0e7c51f317a3c9f/users/7b2277c33fa740f4ba98f44258849ad1/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:01.251035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d60aa4-5c0b-4300-bfac-bf9dac6d0956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc1ccdeeb0274b89a0e7c51f317a3c9f, user_id=7b2277c33fa740f4ba98f44258849ad1, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:01.260079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d60aa4-5c0b-4300-bfac-bf9dac6d0956 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:01.277200 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 877/1695] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:01 2022] PUT /identity/v3/projects/fc1ccdeeb0274b89a0e7c51f317a3c9f/users/7b2277c33fa740f4ba98f44258849ad1/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:01.290089 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30012cdb-af76-43ae-9c05-ce4d56a2aa4e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:01.290443 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30012cdb-af76-43ae-9c05-ce4d56a2aa4e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:01.290637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30012cdb-af76-43ae-9c05-ce4d56a2aa4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:01.365646 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-30012cdb-af76-43ae-9c05-ce4d56a2aa4e None None] MFA Rules not processed for user `7b2277c33fa740f4ba98f44258849ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:01.369191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-30012cdb-af76-43ae-9c05-ce4d56a2aa4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:01.466741 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:01.534941 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 819/1696] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:01.541620 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d60aa4-5c0b-4300-bfac-bf9dac6d0956 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:01.544562 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-36d60aa4-5c0b-4300-bfac-bf9dac6d0956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:01.636860 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:01.698659 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826a612-9670-4ba0-9546-f62f18302817 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:01.698949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826a612-9670-4ba0-9546-f62f18302817 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:01.699221 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826a612-9670-4ba0-9546-f62f18302817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:01.723885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d60aa4-5c0b-4300-bfac-bf9dac6d0956 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:01.738530 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2542a5-0c81-4884-b3d4-a2c9b94968ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:01.739178 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2542a5-0c81-4884-b3d4-a2c9b94968ee service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:01.739459 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2542a5-0c81-4884-b3d4-a2c9b94968ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:01.739732 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2542a5-0c81-4884-b3d4-a2c9b94968ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:01.761832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b826a612-9670-4ba0-9546-f62f18302817 None None] MFA Rules not processed for user `03117ffa81334df8a3467adbfaf96059`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:01.776873 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2542a5-0c81-4884-b3d4-a2c9b94968ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:01.779280 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2542a5-0c81-4884-b3d4-a2c9b94968ee service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:01.783879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b826a612-9670-4ba0-9546-f62f18302817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:01.821922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 878/1697] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:33:01.982753 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 820/1698] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:01.993865 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da0e74-49fa-4ee3-a2f3-79fe55751b3e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:01.994258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da0e74-49fa-4ee3-a2f3-79fe55751b3e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:01.994669 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da0e74-49fa-4ee3-a2f3-79fe55751b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:02.005695 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826a612-9670-4ba0-9546-f62f18302817 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:02.007612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b826a612-9670-4ba0-9546-f62f18302817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:02.034656 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826a612-9670-4ba0-9546-f62f18302817 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:02.038765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-c9da0e74-49fa-4ee3-a2f3-79fe55751b3e None None] MFA Rules not processed for user `03117ffa81334df8a3467adbfaf96059`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:02.042079 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02180d12-700c-43cd-b577-198170d69b55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:02.042922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c9da0e74-49fa-4ee3-a2f3-79fe55751b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:02.045292 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02180d12-700c-43cd-b577-198170d69b55 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:02.045510 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02180d12-700c-43cd-b577-198170d69b55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:02.045796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02180d12-700c-43cd-b577-198170d69b55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:02.061911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 879/1699] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:02.075910 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da0e74-49fa-4ee3-a2f3-79fe55751b3e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:02.078685 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c9da0e74-49fa-4ee3-a2f3-79fe55751b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:02.208155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02180d12-700c-43cd-b577-198170d69b55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:02.209559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02180d12-700c-43cd-b577-198170d69b55 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:02.239351 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 821/1700] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:33:02.252608 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da0e74-49fa-4ee3-a2f3-79fe55751b3e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:02.261956 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ba4a8-3e5e-4b85-b744-53707b3c46c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:02.263056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ba4a8-3e5e-4b85-b744-53707b3c46c2 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:02.263589 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ba4a8-3e5e-4b85-b744-53707b3c46c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:02.264048 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ba4a8-3e5e-4b85-b744-53707b3c46c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:02.282243 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ba4a8-3e5e-4b85-b744-53707b3c46c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:02.287008 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ba4a8-3e5e-4b85-b744-53707b3c46c2 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:02.319414 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 880/1701] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:33:02.360521 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02180d12-700c-43cd-b577-198170d69b55 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:02.363327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-02180d12-700c-43cd-b577-198170d69b55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:02.439578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfa26c-10ee-48e9-8528-60bd37c73d07 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:02.440286 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfa26c-10ee-48e9-8528-60bd37c73d07 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:02.440560 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfa26c-10ee-48e9-8528-60bd37c73d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:02.526446 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-e9bfa26c-10ee-48e9-8528-60bd37c73d07 None None] MFA Rules not processed for user `8fbc5bd64315421e97931402fb0e3a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:02.532838 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e9bfa26c-10ee-48e9-8528-60bd37c73d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:02.580236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02180d12-700c-43cd-b577-198170d69b55 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:02.627455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0eb83-04a6-479f-a0c4-27f3f8f75ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:02.628441 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0eb83-04a6-479f-a0c4-27f3f8f75ef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:02.628737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0eb83-04a6-479f-a0c4-27f3f8f75ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:02.629077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0eb83-04a6-479f-a0c4-27f3f8f75ef2 admin admin] PATH_INFO: `/v3/projects/fcb16ab0cf4a4428ade065b9f7468a5e` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:02.636544 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa0eb83-04a6-479f-a0c4-27f3f8f75ef2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcb16ab0cf4a4428ade065b9f7468a5e)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:02.641659 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa0eb83-04a6-479f-a0c4-27f3f8f75ef2 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:02.653075 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:02.718243 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 881/1702] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:02.719828 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8081896-7b8d-491c-a872-d7ea16a64e69 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:02.720147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8081896-7b8d-491c-a872-d7ea16a64e69 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:02.720487 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8081896-7b8d-491c-a872-d7ea16a64e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:02.772324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-c8081896-7b8d-491c-a872-d7ea16a64e69 None None] MFA Rules not processed for user `8fbc5bd64315421e97931402fb0e3a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:02.773382 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c8081896-7b8d-491c-a872-d7ea16a64e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:02.779669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-afa0eb83-04a6-479f-a0c4-27f3f8f75ef2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcb16ab0cf4a4428ade065b9f7468a5e', 'request_id': 'req-afa0eb83-04a6-479f-a0c4-27f3f8f75ef2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:02.796179 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-afa0eb83-04a6-479f-a0c4-27f3f8f75ef2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcb16ab0cf4a4428ade065b9f7468a5e', 'request_id': 'req-afa0eb83-04a6-479f-a0c4-27f3f8f75ef2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:02.814535 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-afa0eb83-04a6-479f-a0c4-27f3f8f75ef2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcb16ab0cf4a4428ade065b9f7468a5e', 'request_id': 'req-afa0eb83-04a6-479f-a0c4-27f3f8f75ef2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:02.841413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-afa0eb83-04a6-479f-a0c4-27f3f8f75ef2 admin admin] The token cache is being invalidate because project fcb16ab0cf4a4428ade065b9f7468a5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:02.844376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-afa0eb83-04a6-479f-a0c4-27f3f8f75ef2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:02.847543 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-afa0eb83-04a6-479f-a0c4-27f3f8f75ef2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:02.852053 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 822/1703] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:02 2022] DELETE /identity/v3/projects/fcb16ab0cf4a4428ade065b9f7468a5e => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:02.856344 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0eb83-04a6-479f-a0c4-27f3f8f75ef2 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:02.857472 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:02.860300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-afa0eb83-04a6-479f-a0c4-27f3f8f75ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:02.915621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 882/1704] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:02.973884 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aeac48-5699-4431-a30c-17fb08924968 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:02.974209 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aeac48-5699-4431-a30c-17fb08924968 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:02.974472 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aeac48-5699-4431-a30c-17fb08924968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:03.012102 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0eb83-04a6-479f-a0c4-27f3f8f75ef2 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:03.018320 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d10329-6238-45b7-af4e-2bf4a7965752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:03.019198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d10329-6238-45b7-af4e-2bf4a7965752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:03.019597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d10329-6238-45b7-af4e-2bf4a7965752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:03.020655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d10329-6238-45b7-af4e-2bf4a7965752 admin admin] PATH_INFO: `/v3/users/da370c57ce8e4c83b54527580f4f70d8` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:03.022354 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d10329-6238-45b7-af4e-2bf4a7965752 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da370c57ce8e4c83b54527580f4f70d8)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:03.024462 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d10329-6238-45b7-af4e-2bf4a7965752 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:03.059690 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-c5aeac48-5699-4431-a30c-17fb08924968 None None] MFA Rules not processed for user `adc5854ed52042369a9b153af7da4cfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:03.061884 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c5aeac48-5699-4431-a30c-17fb08924968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:03.147251 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d5d10329-6238-45b7-af4e-2bf4a7965752 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da370c57ce8e4c83b54527580f4f70d8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:03.170134 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d5d10329-6238-45b7-af4e-2bf4a7965752 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da370c57ce8e4c83b54527580f4f70d8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:03.191214 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d5d10329-6238-45b7-af4e-2bf4a7965752 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da370c57ce8e4c83b54527580f4f70d8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:03.192242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d5d10329-6238-45b7-af4e-2bf4a7965752 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da370c57ce8e4c83b54527580f4f70d8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:03.208496 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 883/1705] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:03.220381 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aeac48-5699-4431-a30c-17fb08924968 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:03.222881 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c5aeac48-5699-4431-a30c-17fb08924968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:03.238599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d5d10329-6238-45b7-af4e-2bf4a7965752 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da370c57ce8e4c83b54527580f4f70d8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:03.259537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 823/1706] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:02 2022] DELETE /identity/v3/users/da370c57ce8e4c83b54527580f4f70d8 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:03.261069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56dfb8-2e7d-4d73-aa47-764d395b8f17 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:03.261359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56dfb8-2e7d-4d73-aa47-764d395b8f17 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:03.261657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56dfb8-2e7d-4d73-aa47-764d395b8f17 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:03.263362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 824/1707] 10.222.0.18 () {54 vars in 912 bytes} [Mon Aug 29 20:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:33:03.273319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed25c98-8472-4f74-8bfb-c0d12caf2c12 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:03.273850 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed25c98-8472-4f74-8bfb-c0d12caf2c12 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:03.274167 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed25c98-8472-4f74-8bfb-c0d12caf2c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:03.334640 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:03.370992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-fed25c98-8472-4f74-8bfb-c0d12caf2c12 None None] MFA Rules not processed for user `d51e331a4c9c4f038bdb80f8c5b74e42`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:03.372993 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-fed25c98-8472-4f74-8bfb-c0d12caf2c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:03.403721 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aeac48-5699-4431-a30c-17fb08924968 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:03.478230 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 825/1708] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 29 20:33:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:03.479288 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed25c98-8472-4f74-8bfb-c0d12caf2c12 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:03.482517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-fed25c98-8472-4f74-8bfb-c0d12caf2c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:03.519646 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed25c98-8472-4f74-8bfb-c0d12caf2c12 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:03.538880 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695d14a-a66e-4abb-9a03-fa3730aaf30a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:03.539523 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695d14a-a66e-4abb-9a03-fa3730aaf30a service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:03.539831 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695d14a-a66e-4abb-9a03-fa3730aaf30a service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:03.544365 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695d14a-a66e-4abb-9a03-fa3730aaf30a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:03.574722 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695d14a-a66e-4abb-9a03-fa3730aaf30a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:03.576402 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695d14a-a66e-4abb-9a03-fa3730aaf30a service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:03.643475 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 826/1709] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:03.654702 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d6fb3-2e31-47f4-bcbf-061ee1023382 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:03.655473 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d6fb3-2e31-47f4-bcbf-061ee1023382 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:03.655780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d6fb3-2e31-47f4-bcbf-061ee1023382 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:03.660042 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d6fb3-2e31-47f4-bcbf-061ee1023382 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:03.669273 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-029d6fb3-2e31-47f4-bcbf-061ee1023382 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:03.709329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:03.848760 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695d14a-a66e-4abb-9a03-fa3730aaf30a service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:03.850717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2695d14a-a66e-4abb-9a03-fa3730aaf30a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:03.857018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d6fb3-2e31-47f4-bcbf-061ee1023382 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:03.860900 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d6fb3-2e31-47f4-bcbf-061ee1023382 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:03.899971 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 884/1710] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:03 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:04.021587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695d14a-a66e-4abb-9a03-fa3730aaf30a service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:04.032637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf44344-3478-4511-8b2f-505a7ac7aba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:04.033314 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf44344-3478-4511-8b2f-505a7ac7aba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:04.033599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf44344-3478-4511-8b2f-505a7ac7aba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:04.033872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf44344-3478-4511-8b2f-505a7ac7aba6 admin admin] PATH_INFO: `/v3/projects/13409c26096a440aae9b98eba1ba0991` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:04.039014 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf44344-3478-4511-8b2f-505a7ac7aba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13409c26096a440aae9b98eba1ba0991)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:04.041348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf44344-3478-4511-8b2f-505a7ac7aba6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:04.170173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7cf44344-3478-4511-8b2f-505a7ac7aba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13409c26096a440aae9b98eba1ba0991', 'request_id': 'req-7cf44344-3478-4511-8b2f-505a7ac7aba6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:04.190513 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7cf44344-3478-4511-8b2f-505a7ac7aba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13409c26096a440aae9b98eba1ba0991', 'request_id': 'req-7cf44344-3478-4511-8b2f-505a7ac7aba6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:04.207490 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7cf44344-3478-4511-8b2f-505a7ac7aba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13409c26096a440aae9b98eba1ba0991', 'request_id': 'req-7cf44344-3478-4511-8b2f-505a7ac7aba6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:04.237878 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7cf44344-3478-4511-8b2f-505a7ac7aba6 admin admin] The token cache is being invalidate because project 13409c26096a440aae9b98eba1ba0991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:04.237878 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7cf44344-3478-4511-8b2f-505a7ac7aba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:04.240834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7cf44344-3478-4511-8b2f-505a7ac7aba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:04.244276 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 827/1711] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:03 2022] DELETE /identity/v3/projects/13409c26096a440aae9b98eba1ba0991 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:04.249569 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b76ff9-1d96-45a5-8c9b-0d48c031e221 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:04.249862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b76ff9-1d96-45a5-8c9b-0d48c031e221 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:04.250179 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b76ff9-1d96-45a5-8c9b-0d48c031e221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:04.287768 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-49b76ff9-1d96-45a5-8c9b-0d48c031e221 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:04.289186 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-49b76ff9-1d96-45a5-8c9b-0d48c031e221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:04.446574 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 885/1712] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:04.455176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3087a0-c6ed-4b85-b9af-a26dee85be9e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:04.455843 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3087a0-c6ed-4b85-b9af-a26dee85be9e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:04.455843 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3087a0-c6ed-4b85-b9af-a26dee85be9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:04.535965 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1a3087a0-c6ed-4b85-b9af-a26dee85be9e None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:04.550301 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1a3087a0-c6ed-4b85-b9af-a26dee85be9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:04.596325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 828/1713] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:04.597726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b76ff9-1d96-45a5-8c9b-0d48c031e221 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:04.631917 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b76ff9-1d96-45a5-8c9b-0d48c031e221 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:04.661146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f439c-5eb6-48c2-8ce4-da8daffb0498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:04.676491 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be0678-a029-414b-b257-16ac179cc2f0 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:04.677008 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be0678-a029-414b-b257-16ac179cc2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:04.677443 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be0678-a029-414b-b257-16ac179cc2f0 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:04.679435 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 829/1714] 10.222.0.18 () {54 vars in 912 bytes} [Mon Aug 29 20:33:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:33:04.694783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f439c-5eb6-48c2-8ce4-da8daffb0498 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:04.709814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f439c-5eb6-48c2-8ce4-da8daffb0498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:04.709814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f439c-5eb6-48c2-8ce4-da8daffb0498 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:04.710553 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f439c-5eb6-48c2-8ce4-da8daffb0498 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:33:04.710883 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f439c-5eb6-48c2-8ce4-da8daffb0498 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:04.719064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87003472-a51b-49a3-88ef-7efcf0ffc8a0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:04.719388 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87003472-a51b-49a3-88ef-7efcf0ffc8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:04.719617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87003472-a51b-49a3-88ef-7efcf0ffc8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:04.729127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f439c-5eb6-48c2-8ce4-da8daffb0498 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:04.756015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 886/1715] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:33:04.757026 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f439c-5eb6-48c2-8ce4-da8daffb0498 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:04.763181 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2c6f439c-5eb6-48c2-8ce4-da8daffb0498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:04.785814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-87003472-a51b-49a3-88ef-7efcf0ffc8a0 None None] MFA Rules not processed for user `d51e331a4c9c4f038bdb80f8c5b74e42`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:04.787267 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-87003472-a51b-49a3-88ef-7efcf0ffc8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:04.814696 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:04.907043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 830/1716] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 29 20:33:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:04.908183 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87003472-a51b-49a3-88ef-7efcf0ffc8a0 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:04.949578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87003472-a51b-49a3-88ef-7efcf0ffc8a0 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:04.951128 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f439c-5eb6-48c2-8ce4-da8daffb0498 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:04.961908 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662cc45-a4d8-4449-a730-970e394836bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:04.966899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662cc45-a4d8-4449-a730-970e394836bb admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:04.967790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662cc45-a4d8-4449-a730-970e394836bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:04.968204 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662cc45-a4d8-4449-a730-970e394836bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:04.969650 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662cc45-a4d8-4449-a730-970e394836bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:04.975866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662cc45-a4d8-4449-a730-970e394836bb admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:04.990921 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9202a7-7554-4809-87b6-502ac40da586 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0e659eee5e4a9da8d3d7337cb73798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:04.991688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9202a7-7554-4809-87b6-502ac40da586 service placement] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:04.992172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9202a7-7554-4809-87b6-502ac40da586 service placement] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:04.992579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9202a7-7554-4809-87b6-502ac40da586 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:04.996360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4f9202a7-7554-4809-87b6-502ac40da586 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:05.041224 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9202a7-7554-4809-87b6-502ac40da586 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:05.058184 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9202a7-7554-4809-87b6-502ac40da586 service placement] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:05.074041 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 831/1717] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:04 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:05.074041 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662cc45-a4d8-4449-a730-970e394836bb admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:05.076945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9662cc45-a4d8-4449-a730-970e394836bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:05.112293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 887/1718] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:05.113262 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9202a7-7554-4809-87b6-502ac40da586 service placement] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:05.144848 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9202a7-7554-4809-87b6-502ac40da586 service placement] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:05.265256 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:05.280612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662cc45-a4d8-4449-a730-970e394836bb admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:05.306596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d505c-074b-4b03-b000-bac84f27b17e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:05.307435 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d505c-074b-4b03-b000-bac84f27b17e service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:05.307875 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d505c-074b-4b03-b000-bac84f27b17e service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:05.308881 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d505c-074b-4b03-b000-bac84f27b17e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:05.344819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d505c-074b-4b03-b000-bac84f27b17e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:05.346530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d505c-074b-4b03-b000-bac84f27b17e service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:05.352510 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb66f7-165d-4c89-8e70-80ac1471e1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:05.353763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb66f7-165d-4c89-8e70-80ac1471e1bc admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:05.353763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb66f7-165d-4c89-8e70-80ac1471e1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:05.353763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb66f7-165d-4c89-8e70-80ac1471e1bc admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:05.358407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb66f7-165d-4c89-8e70-80ac1471e1bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:05.364623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb66f7-165d-4c89-8e70-80ac1471e1bc admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:05.374145 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 832/1719] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:05.419855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 888/1720] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:05 2022] POST /identity/v3/users => generated 294 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:05.425131 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d505c-074b-4b03-b000-bac84f27b17e service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:05.443383 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d505c-074b-4b03-b000-bac84f27b17e service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:05.460279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73a00f-e79d-4f94-a54f-4bbad0e985cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:05.461088 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73a00f-e79d-4f94-a54f-4bbad0e985cc admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:05.461541 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73a00f-e79d-4f94-a54f-4bbad0e985cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:05.461969 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d73a00f-e79d-4f94-a54f-4bbad0e985cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:05.462767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73a00f-e79d-4f94-a54f-4bbad0e985cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:05.464594 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d73a00f-e79d-4f94-a54f-4bbad0e985cc admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:05.490266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 833/1721] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:05.492955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb66f7-165d-4c89-8e70-80ac1471e1bc admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:05.520249 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb66f7-165d-4c89-8e70-80ac1471e1bc admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:05.531177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa846368-a1d9-410e-80b8-6ca1af658f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:05.539195 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa846368-a1d9-410e-80b8-6ca1af658f38 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:05.539731 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa846368-a1d9-410e-80b8-6ca1af658f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:05.544996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa846368-a1d9-410e-80b8-6ca1af658f38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:05.546017 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa846368-a1d9-410e-80b8-6ca1af658f38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:05.556002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa846368-a1d9-410e-80b8-6ca1af658f38 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:05.598356 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73a00f-e79d-4f94-a54f-4bbad0e985cc admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:05.598668 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 889/1722] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:05.615780 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d73a00f-e79d-4f94-a54f-4bbad0e985cc admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:05.627039 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c51ed-1259-47be-b039-c92bb22d92e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:05.628910 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c51ed-1259-47be-b039-c92bb22d92e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:05.629560 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c51ed-1259-47be-b039-c92bb22d92e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:05.630055 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c51ed-1259-47be-b039-c92bb22d92e6 admin admin] PATH_INFO: `/v3/projects/908ee0fecf5a4a078fddabb7e8179fe0/users/0426d613bd51455792b6e52fe22bfd7b/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:05.654275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c51ed-1259-47be-b039-c92bb22d92e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=908ee0fecf5a4a078fddabb7e8179fe0, user_id=0426d613bd51455792b6e52fe22bfd7b, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:05.658242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c51ed-1259-47be-b039-c92bb22d92e6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:05.683755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 834/1723] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:05 2022] PUT /identity/v3/projects/908ee0fecf5a4a078fddabb7e8179fe0/users/0426d613bd51455792b6e52fe22bfd7b/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:05.696830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755d4c6-ecd0-494e-aa1d-8bf80ee6f0c9 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:05.697125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755d4c6-ecd0-494e-aa1d-8bf80ee6f0c9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:05.697425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755d4c6-ecd0-494e-aa1d-8bf80ee6f0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:05.787028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-7755d4c6-ecd0-494e-aa1d-8bf80ee6f0c9 None None] MFA Rules not processed for user `0426d613bd51455792b6e52fe22bfd7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:05.791064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7755d4c6-ecd0-494e-aa1d-8bf80ee6f0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:06.027260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c51ed-1259-47be-b039-c92bb22d92e6 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:06.046464 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 890/1724] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:06.144030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c51ed-1259-47be-b039-c92bb22d92e6 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:06.174477 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7551c-a553-4dee-a0b1-a8e03b54034a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:06.176653 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7551c-a553-4dee-a0b1-a8e03b54034a admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:06.179709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7551c-a553-4dee-a0b1-a8e03b54034a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:06.180850 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7551c-a553-4dee-a0b1-a8e03b54034a admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:06.184239 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7551c-a553-4dee-a0b1-a8e03b54034a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:06.195898 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7551c-a553-4dee-a0b1-a8e03b54034a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:06.264030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:06.308134 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 835/1725] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:06 2022] POST /identity/v3/projects => generated 385 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:06.309426 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7755d4c6-ecd0-494e-aa1d-8bf80ee6f0c9 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:06.357146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7755d4c6-ecd0-494e-aa1d-8bf80ee6f0c9 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:06.542699 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fba3a7-9a70-4f2c-ac14-4a0f4a9418f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:06.545112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fba3a7-9a70-4f2c-ac14-4a0f4a9418f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:06.545405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fba3a7-9a70-4f2c-ac14-4a0f4a9418f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:06.545688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fba3a7-9a70-4f2c-ac14-4a0f4a9418f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:06.547546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fba3a7-9a70-4f2c-ac14-4a0f4a9418f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:06.556065 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fba3a7-9a70-4f2c-ac14-4a0f4a9418f6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:06.627099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 891/1726] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:06 2022] POST /identity/v3/users => generated 293 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:06.628313 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7551c-a553-4dee-a0b1-a8e03b54034a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:06.655548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7551c-a553-4dee-a0b1-a8e03b54034a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:06.686096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c61c7-ddad-4652-b037-9bb1f56033c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:06.686723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c61c7-ddad-4652-b037-9bb1f56033c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:06.687008 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c61c7-ddad-4652-b037-9bb1f56033c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:06.687306 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c61c7-ddad-4652-b037-9bb1f56033c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:06.704200 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c61c7-ddad-4652-b037-9bb1f56033c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:06.711868 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c61c7-ddad-4652-b037-9bb1f56033c2 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:06.820115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 836/1727] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:06.834042 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fba3a7-9a70-4f2c-ac14-4a0f4a9418f6 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:06.857297 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fba3a7-9a70-4f2c-ac14-4a0f4a9418f6 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:06.890391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b158eb-07cb-470b-85d8-dab9a9d15074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:06.891957 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b158eb-07cb-470b-85d8-dab9a9d15074 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:06.892244 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b158eb-07cb-470b-85d8-dab9a9d15074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:06.892515 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b158eb-07cb-470b-85d8-dab9a9d15074 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:06.893233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b158eb-07cb-470b-85d8-dab9a9d15074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:06.903124 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b158eb-07cb-470b-85d8-dab9a9d15074 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:06.950197 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 892/1728] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:06.954358 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c61c7-ddad-4652-b037-9bb1f56033c2 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:06.982963 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c61c7-ddad-4652-b037-9bb1f56033c2 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:07.001804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60402b51-2577-43db-9872-caf581bddd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:07.004688 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60402b51-2577-43db-9872-caf581bddd4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:07.004984 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60402b51-2577-43db-9872-caf581bddd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:07.005944 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60402b51-2577-43db-9872-caf581bddd4b admin admin] PATH_INFO: `/v3/projects/eb689752adfe4c92aaa6e5e9bbf25a6b/users/f79ca9c2a9274d668ac5a67ae04890f0/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:07.077782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60402b51-2577-43db-9872-caf581bddd4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb689752adfe4c92aaa6e5e9bbf25a6b, user_id=f79ca9c2a9274d668ac5a67ae04890f0, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:07.108390 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60402b51-2577-43db-9872-caf581bddd4b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:07.122993 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b158eb-07cb-470b-85d8-dab9a9d15074 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:07.124914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-44b158eb-07cb-470b-85d8-dab9a9d15074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:07.173821 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b158eb-07cb-470b-85d8-dab9a9d15074 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:07.193536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 837/1729] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:06 2022] PUT /identity/v3/projects/eb689752adfe4c92aaa6e5e9bbf25a6b/users/f79ca9c2a9274d668ac5a67ae04890f0/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:07.197184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab8841-2a43-4a70-88e3-e7c5fb8789d4 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:07.197940 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab8841-2a43-4a70-88e3-e7c5fb8789d4 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:07.198529 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab8841-2a43-4a70-88e3-e7c5fb8789d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:07.245970 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:07.258731 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-70ab8841-2a43-4a70-88e3-e7c5fb8789d4 None None] MFA Rules not processed for user `8fbc5bd64315421e97931402fb0e3a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:07.260627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-70ab8841-2a43-4a70-88e3-e7c5fb8789d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:07.356468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021003db-57b1-447e-a60e-d26fc2fa055c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:07.361389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021003db-57b1-447e-a60e-d26fc2fa055c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:07.361687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021003db-57b1-447e-a60e-d26fc2fa055c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:07.368120 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021003db-57b1-447e-a60e-d26fc2fa055c admin admin] PATH_INFO: `/v3/users/18b635c193494d43adc087175b4eed4d` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:07.377074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021003db-57b1-447e-a60e-d26fc2fa055c admin admin] RBAC: Authorizing `identity:delete_user(user_id=18b635c193494d43adc087175b4eed4d)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:07.379065 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021003db-57b1-447e-a60e-d26fc2fa055c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:07.402821 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 838/1730] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:07.405411 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f89cd-68e4-4aa9-b738-34ccad694ea0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:07.406030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f89cd-68e4-4aa9-b738-34ccad694ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:07.407262 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f89cd-68e4-4aa9-b738-34ccad694ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:07.476298 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-8c3f89cd-68e4-4aa9-b738-34ccad694ea0 None None] MFA Rules not processed for user `f79ca9c2a9274d668ac5a67ae04890f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:07.479378 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8c3f89cd-68e4-4aa9-b738-34ccad694ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:07.534331 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-021003db-57b1-447e-a60e-d26fc2fa055c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18b635c193494d43adc087175b4eed4d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:07.556388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-021003db-57b1-447e-a60e-d26fc2fa055c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18b635c193494d43adc087175b4eed4d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:07.580379 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-021003db-57b1-447e-a60e-d26fc2fa055c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18b635c193494d43adc087175b4eed4d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:07.581179 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-021003db-57b1-447e-a60e-d26fc2fa055c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18b635c193494d43adc087175b4eed4d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:07.587852 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-021003db-57b1-447e-a60e-d26fc2fa055c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18b635c193494d43adc087175b4eed4d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:07.652592 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 893/1731] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:07 2022] DELETE /identity/v3/users/18b635c193494d43adc087175b4eed4d => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:07.653917 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021003db-57b1-447e-a60e-d26fc2fa055c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:07.660182 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-021003db-57b1-447e-a60e-d26fc2fa055c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:07.685350 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 839/1732] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:07.690766 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52592159-d79f-4810-b1d7-3cba26a4f4d0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:07.691409 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52592159-d79f-4810-b1d7-3cba26a4f4d0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:07.694578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52592159-d79f-4810-b1d7-3cba26a4f4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:07.759421 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-52592159-d79f-4810-b1d7-3cba26a4f4d0 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:07.765938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-52592159-d79f-4810-b1d7-3cba26a4f4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:07.821526 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021003db-57b1-447e-a60e-d26fc2fa055c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:07.831337 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c19af-645e-4447-b066-3594a77fa067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:07.831923 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c19af-645e-4447-b066-3594a77fa067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:07.832310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c19af-645e-4447-b066-3594a77fa067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:07.832591 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c19af-645e-4447-b066-3594a77fa067 admin admin] PATH_INFO: `/v3/users/e5e618cf472e4275bec4d6222fa29ebe` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:07.838653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c19af-645e-4447-b066-3594a77fa067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5e618cf472e4275bec4d6222fa29ebe)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:07.841067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c19af-645e-4447-b066-3594a77fa067 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:07.866719 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:07.900481 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 840/1733] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:07.901910 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52592159-d79f-4810-b1d7-3cba26a4f4d0 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:07.905061 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-52592159-d79f-4810-b1d7-3cba26a4f4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:07.933753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:07.936216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52592159-d79f-4810-b1d7-3cba26a4f4d0 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:07.941399 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6b8c19af-645e-4447-b066-3594a77fa067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5e618cf472e4275bec4d6222fa29ebe'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:07.949614 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ef697-89fc-4669-a689-2cb70efa0e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:07.950946 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ef697-89fc-4669-a689-2cb70efa0e64 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:07.951537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ef697-89fc-4669-a689-2cb70efa0e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:07.952233 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ef697-89fc-4669-a689-2cb70efa0e64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:07.953758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ef697-89fc-4669-a689-2cb70efa0e64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:07.957689 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ef697-89fc-4669-a689-2cb70efa0e64 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:07.959019 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6b8c19af-645e-4447-b066-3594a77fa067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5e618cf472e4275bec4d6222fa29ebe'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:07.979164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6b8c19af-645e-4447-b066-3594a77fa067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5e618cf472e4275bec4d6222fa29ebe'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:07.981083 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6b8c19af-645e-4447-b066-3594a77fa067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5e618cf472e4275bec4d6222fa29ebe'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:07.988043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6b8c19af-645e-4447-b066-3594a77fa067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5e618cf472e4275bec4d6222fa29ebe'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:08.056062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 841/1734] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:07 2022] POST /identity/v3/projects => generated 387 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:08.056242 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 894/1735] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:07 2022] DELETE /identity/v3/users/e5e618cf472e4275bec4d6222fa29ebe => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:08.056360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ef697-89fc-4669-a689-2cb70efa0e64 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:08.059548 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d76e0b-47e2-4b4c-a52c-f91062d7128e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:08.059842 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d76e0b-47e2-4b4c-a52c-f91062d7128e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:08.060153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d76e0b-47e2-4b4c-a52c-f91062d7128e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:08.064836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-fd9ef697-89fc-4669-a689-2cb70efa0e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:08.098189 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-75d76e0b-47e2-4b4c-a52c-f91062d7128e None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:08.100677 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-75d76e0b-47e2-4b4c-a52c-f91062d7128e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:08.152912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ef697-89fc-4669-a689-2cb70efa0e64 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:08.165423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09862209-068b-4e1a-bfff-a2da596a954e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:08.165749 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09862209-068b-4e1a-bfff-a2da596a954e service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:08.166034 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09862209-068b-4e1a-bfff-a2da596a954e service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:08.166309 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09862209-068b-4e1a-bfff-a2da596a954e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:08.168274 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-09862209-068b-4e1a-bfff-a2da596a954e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:08.225687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 895/1736] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:08.225687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d76e0b-47e2-4b4c-a52c-f91062d7128e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:08.225687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-75d76e0b-47e2-4b4c-a52c-f91062d7128e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:08.243448 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d76e0b-47e2-4b4c-a52c-f91062d7128e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:08.258242 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1e2df-2ee0-4b93-907e-d246491da1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:08.259013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1e2df-2ee0-4b93-907e-d246491da1c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:08.260365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1e2df-2ee0-4b93-907e-d246491da1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:08.262442 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1e2df-2ee0-4b93-907e-d246491da1c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:08.263210 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc1e2df-2ee0-4b93-907e-d246491da1c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:08.271348 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc1e2df-2ee0-4b93-907e-d246491da1c1 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:08.317827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 896/1737] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:08 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:08.318984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1e2df-2ee0-4b93-907e-d246491da1c1 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:08.326817 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09862209-068b-4e1a-bfff-a2da596a954e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:08.329202 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09862209-068b-4e1a-bfff-a2da596a954e service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:08.344326 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1e2df-2ee0-4b93-907e-d246491da1c1 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:08.349815 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 842/1738] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:08.350585 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09862209-068b-4e1a-bfff-a2da596a954e service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:08.352370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1becd-a610-4923-86a0-bd86dda3863f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:08.355030 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1becd-a610-4923-86a0-bd86dda3863f service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:08.355538 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1becd-a610-4923-86a0-bd86dda3863f service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:08.356097 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1becd-a610-4923-86a0-bd86dda3863f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:08.379675 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1becd-a610-4923-86a0-bd86dda3863f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:08.383993 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1becd-a610-4923-86a0-bd86dda3863f service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:08.393950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09862209-068b-4e1a-bfff-a2da596a954e service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:08.421545 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f9d74-3389-4dbf-a281-b88410eb8dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:08.422270 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f9d74-3389-4dbf-a281-b88410eb8dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:08.423182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f9d74-3389-4dbf-a281-b88410eb8dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:08.423531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f9d74-3389-4dbf-a281-b88410eb8dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:08.424282 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f9d74-3389-4dbf-a281-b88410eb8dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:08.425976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314f9d74-3389-4dbf-a281-b88410eb8dc9 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:08.435074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 897/1739] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:08.465773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 843/1740] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:08.470926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1becd-a610-4923-86a0-bd86dda3863f service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:08.485777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1becd-a610-4923-86a0-bd86dda3863f service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:08.492703 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cc8a4-5a1b-4f54-9162-4f313fd1ecf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:08.493691 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cc8a4-5a1b-4f54-9162-4f313fd1ecf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:08.494205 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cc8a4-5a1b-4f54-9162-4f313fd1ecf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:08.494472 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cc8a4-5a1b-4f54-9162-4f313fd1ecf8 admin admin] PATH_INFO: `/v3/projects/57d5176d914d4981ba9042a76aacf2c3/users/5245be497c1a4cf9a32e33ed54727442/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:08.537279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2cc8a4-5a1b-4f54-9162-4f313fd1ecf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57d5176d914d4981ba9042a76aacf2c3, user_id=5245be497c1a4cf9a32e33ed54727442, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:08.540335 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2cc8a4-5a1b-4f54-9162-4f313fd1ecf8 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:08.577351 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 898/1741] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:08 2022] PUT /identity/v3/projects/57d5176d914d4981ba9042a76aacf2c3/users/5245be497c1a4cf9a32e33ed54727442/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:08.577814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f9d74-3389-4dbf-a281-b88410eb8dc9 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:08.592136 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f9d74-3389-4dbf-a281-b88410eb8dc9 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:08.726932 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a97fdc3-512d-472f-9102-e6bb2680d5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:08.727584 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97fdc3-512d-472f-9102-e6bb2680d5aa admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:08.727905 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97fdc3-512d-472f-9102-e6bb2680d5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:08.728290 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97fdc3-512d-472f-9102-e6bb2680d5aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:08.728964 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a97fdc3-512d-472f-9102-e6bb2680d5aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:08.730671 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a97fdc3-512d-472f-9102-e6bb2680d5aa admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:08.758543 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 844/1742] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:08.764288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cc8a4-5a1b-4f54-9162-4f313fd1ecf8 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:08.816026 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cc8a4-5a1b-4f54-9162-4f313fd1ecf8 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:08.831607 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6609820-101f-4b5e-b2b8-15642acbc5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:08.833602 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6609820-101f-4b5e-b2b8-15642acbc5fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:08.834032 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6609820-101f-4b5e-b2b8-15642acbc5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:08.834678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6609820-101f-4b5e-b2b8-15642acbc5fe admin admin] PATH_INFO: `/v3/projects/57d5176d914d4981ba9042a76aacf2c3/users/5245be497c1a4cf9a32e33ed54727442/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:08.839917 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6609820-101f-4b5e-b2b8-15642acbc5fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=57d5176d914d4981ba9042a76aacf2c3, user_id=5245be497c1a4cf9a32e33ed54727442, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:08.846602 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6609820-101f-4b5e-b2b8-15642acbc5fe admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:08.912707 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 899/1743] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:08 2022] PUT /identity/v3/projects/57d5176d914d4981ba9042a76aacf2c3/users/5245be497c1a4cf9a32e33ed54727442/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:08.928308 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e1b7f-a1a1-4d3d-ba4d-dfdee41cabc7 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:08.928597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e1b7f-a1a1-4d3d-ba4d-dfdee41cabc7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:08.929029 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e1b7f-a1a1-4d3d-ba4d-dfdee41cabc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:08.975047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:09.001782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-786e1b7f-a1a1-4d3d-ba4d-dfdee41cabc7 None None] MFA Rules not processed for user `5245be497c1a4cf9a32e33ed54727442`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:09.004143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-786e1b7f-a1a1-4d3d-ba4d-dfdee41cabc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:09.174546 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 845/1744] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:08 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:09.183685 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6609820-101f-4b5e-b2b8-15642acbc5fe admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:09.185911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c6609820-101f-4b5e-b2b8-15642acbc5fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:09.208208 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e1b7f-a1a1-4d3d-ba4d-dfdee41cabc7 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:09.210857 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-786e1b7f-a1a1-4d3d-ba4d-dfdee41cabc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:09.291411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6609820-101f-4b5e-b2b8-15642acbc5fe admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:09.295605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd99fab-ccd0-4580-b305-ec4345242a3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:09.296508 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd99fab-ccd0-4580-b305-ec4345242a3e service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:09.296843 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd99fab-ccd0-4580-b305-ec4345242a3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:09.297115 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd99fab-ccd0-4580-b305-ec4345242a3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:09.306451 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:DELETE=4,keystone:SELECT=90,keystone:INSERT=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:09.309652 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd99fab-ccd0-4580-b305-ec4345242a3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:09.312434 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd99fab-ccd0-4580-b305-ec4345242a3e service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:09.344184 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 900/1745] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:09 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:09.344347 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd99fab-ccd0-4580-b305-ec4345242a3e service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:09.349191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0bd99fab-ccd0-4580-b305-ec4345242a3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:09.423663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e1b7f-a1a1-4d3d-ba4d-dfdee41cabc7 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:09.445507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a979ba6-3c53-4915-ac33-5a54b40275f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:09.446487 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a979ba6-3c53-4915-ac33-5a54b40275f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:09.446867 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a979ba6-3c53-4915-ac33-5a54b40275f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:09.447421 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a979ba6-3c53-4915-ac33-5a54b40275f3 admin admin] PATH_INFO: `/v3/projects/20abd4fa69204e62b7a49322b31d958f` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:09.449036 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a979ba6-3c53-4915-ac33-5a54b40275f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20abd4fa69204e62b7a49322b31d958f)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:09.453234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a979ba6-3c53-4915-ac33-5a54b40275f3 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:09.504958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd99fab-ccd0-4580-b305-ec4345242a3e service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:09.518761 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771de598-e20e-4848-afdb-992a72f4ba95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:09.519457 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771de598-e20e-4848-afdb-992a72f4ba95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:09.519745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771de598-e20e-4848-afdb-992a72f4ba95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:09.520597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771de598-e20e-4848-afdb-992a72f4ba95 admin admin] PATH_INFO: `/v3/projects/c6a65050f39e49349b0eab86031bab9f` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:09.525610 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771de598-e20e-4848-afdb-992a72f4ba95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6a65050f39e49349b0eab86031bab9f)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:09.527557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771de598-e20e-4848-afdb-992a72f4ba95 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:09.561607 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7a979ba6-3c53-4915-ac33-5a54b40275f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20abd4fa69204e62b7a49322b31d958f', 'request_id': 'req-7a979ba6-3c53-4915-ac33-5a54b40275f3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:09.569836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7a979ba6-3c53-4915-ac33-5a54b40275f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20abd4fa69204e62b7a49322b31d958f', 'request_id': 'req-7a979ba6-3c53-4915-ac33-5a54b40275f3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:09.592303 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7a979ba6-3c53-4915-ac33-5a54b40275f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20abd4fa69204e62b7a49322b31d958f', 'request_id': 'req-7a979ba6-3c53-4915-ac33-5a54b40275f3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:09.617332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7a979ba6-3c53-4915-ac33-5a54b40275f3 admin admin] The token cache is being invalidate because project 20abd4fa69204e62b7a49322b31d958f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:09.618206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7a979ba6-3c53-4915-ac33-5a54b40275f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:09.619250 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7a979ba6-3c53-4915-ac33-5a54b40275f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:09.624917 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 846/1746] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:09 2022] DELETE /identity/v3/projects/20abd4fa69204e62b7a49322b31d958f => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:09.627340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf30f5-e77a-44ba-a0ef-75efe83ac062 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:09.627903 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf30f5-e77a-44ba-a0ef-75efe83ac062 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:09.628115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf30f5-e77a-44ba-a0ef-75efe83ac062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:09.673779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-771de598-e20e-4848-afdb-992a72f4ba95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6a65050f39e49349b0eab86031bab9f', 'request_id': 'req-771de598-e20e-4848-afdb-992a72f4ba95'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:09.675483 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5faf30f5-e77a-44ba-a0ef-75efe83ac062 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:09.676876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5faf30f5-e77a-44ba-a0ef-75efe83ac062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:09.684040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-771de598-e20e-4848-afdb-992a72f4ba95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6a65050f39e49349b0eab86031bab9f', 'request_id': 'req-771de598-e20e-4848-afdb-992a72f4ba95'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:09.697360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-771de598-e20e-4848-afdb-992a72f4ba95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6a65050f39e49349b0eab86031bab9f', 'request_id': 'req-771de598-e20e-4848-afdb-992a72f4ba95'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:09.712960 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-771de598-e20e-4848-afdb-992a72f4ba95 admin admin] The token cache is being invalidate because project c6a65050f39e49349b0eab86031bab9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:09.713780 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:09.714693 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-771de598-e20e-4848-afdb-992a72f4ba95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:09.715660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-771de598-e20e-4848-afdb-992a72f4ba95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:09.717671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 901/1747] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:09 2022] DELETE /identity/v3/projects/c6a65050f39e49349b0eab86031bab9f => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:09.722849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa08ef7-6f80-461e-b45e-0f5df172a566 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:09.723517 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa08ef7-6f80-461e-b45e-0f5df172a566 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:09.723899 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa08ef7-6f80-461e-b45e-0f5df172a566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:09.746306 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-daa08ef7-6f80-461e-b45e-0f5df172a566 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:09.747583 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-daa08ef7-6f80-461e-b45e-0f5df172a566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:09.787277 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 847/1748] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:09.795164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0945f-81c0-483e-9298-53205a9d1678 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:09.796089 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0945f-81c0-483e-9298-53205a9d1678 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:09.796375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa0945f-81c0-483e-9298-53205a9d1678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:09.826213 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-3aa0945f-81c0-483e-9298-53205a9d1678 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:09.827352 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-3aa0945f-81c0-483e-9298-53205a9d1678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:09.838761 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 848/1749] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:09.848396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 902/1750] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:09.849876 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c2bc2-42a1-475c-8eda-4f8f0f480663 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:09.850171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c2bc2-42a1-475c-8eda-4f8f0f480663 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:09.851010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0945f-81c0-483e-9298-53205a9d1678 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:09.854475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c2bc2-42a1-475c-8eda-4f8f0f480663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:09.866893 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa0945f-81c0-483e-9298-53205a9d1678 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:09.873781 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da61aff-a362-4d5a-a19f-61fd7ff4e384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:09.874548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da61aff-a362-4d5a-a19f-61fd7ff4e384 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:09.875112 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da61aff-a362-4d5a-a19f-61fd7ff4e384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:09.875407 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da61aff-a362-4d5a-a19f-61fd7ff4e384 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:09.876564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da61aff-a362-4d5a-a19f-61fd7ff4e384 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:33:09.876952 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da61aff-a362-4d5a-a19f-61fd7ff4e384 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:09.879318 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da61aff-a362-4d5a-a19f-61fd7ff4e384 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:09.881457 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f08c2bc2-42a1-475c-8eda-4f8f0f480663 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:09.882893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f08c2bc2-42a1-475c-8eda-4f8f0f480663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:09.900129 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 903/1751] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:09.900725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 849/1752] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:33:09.901248 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c2bc2-42a1-475c-8eda-4f8f0f480663 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:09.901624 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da61aff-a362-4d5a-a19f-61fd7ff4e384 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:09.916557 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da61aff-a362-4d5a-a19f-61fd7ff4e384 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:09.924828 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c2bc2-42a1-475c-8eda-4f8f0f480663 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:09.925022 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624adbe9-a9ee-4e96-83ee-92afde9f687c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:09.925475 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624adbe9-a9ee-4e96-83ee-92afde9f687c admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:09.925838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624adbe9-a9ee-4e96-83ee-92afde9f687c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:09.926029 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624adbe9-a9ee-4e96-83ee-92afde9f687c admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:09.926685 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624adbe9-a9ee-4e96-83ee-92afde9f687c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:09.929754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624adbe9-a9ee-4e96-83ee-92afde9f687c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:09.931617 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd5859-fe0b-44f4-95f0-b3c050879f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:09.932421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd5859-fe0b-44f4-95f0-b3c050879f12 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:09.932583 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd5859-fe0b-44f4-95f0-b3c050879f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:09.932831 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd5859-fe0b-44f4-95f0-b3c050879f12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:09.933491 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd5859-fe0b-44f4-95f0-b3c050879f12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:33:09.933930 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd5859-fe0b-44f4-95f0-b3c050879f12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:09.935389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd5859-fe0b-44f4-95f0-b3c050879f12 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:09.953353 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 904/1753] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:33:09.959290 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd5859-fe0b-44f4-95f0-b3c050879f12 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:09.974326 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd5859-fe0b-44f4-95f0-b3c050879f12 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:09.981126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d55d54-f127-45e9-b24f-5dc76418fd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:09.981842 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d55d54-f127-45e9-b24f-5dc76418fd65 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:09.982424 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d55d54-f127-45e9-b24f-5dc76418fd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:09.982750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d55d54-f127-45e9-b24f-5dc76418fd65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:09.983411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d55d54-f127-45e9-b24f-5dc76418fd65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:09.985705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d55d54-f127-45e9-b24f-5dc76418fd65 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:09.991430 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:09.999472 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 850/1754] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:09 2022] POST /identity/v3/projects => generated 405 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:10.001874 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624adbe9-a9ee-4e96-83ee-92afde9f687c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:10.033824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 905/1755] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:09 2022] POST /identity/v3/projects => generated 365 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:10.038278 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d55d54-f127-45e9-b24f-5dc76418fd65 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:10.104129 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624adbe9-a9ee-4e96-83ee-92afde9f687c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:10.170060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d55d54-f127-45e9-b24f-5dc76418fd65 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:10.192086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58117e-a276-483d-a680-10a9413b4a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:10.193334 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58117e-a276-483d-a680-10a9413b4a48 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:10.195170 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58117e-a276-483d-a680-10a9413b4a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:10.195686 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58117e-a276-483d-a680-10a9413b4a48 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:10.196830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c58117e-a276-483d-a680-10a9413b4a48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:10.200758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c58117e-a276-483d-a680-10a9413b4a48 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:10.278717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42509091-e801-490b-b753-8ce1271d7bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:10.279743 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42509091-e801-490b-b753-8ce1271d7bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:10.280501 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42509091-e801-490b-b753-8ce1271d7bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:10.281818 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42509091-e801-490b-b753-8ce1271d7bc2 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:10.283680 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42509091-e801-490b-b753-8ce1271d7bc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:10.285478 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 906/1756] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:10 2022] POST /identity/v3/users => generated 283 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:10.286532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58117e-a276-483d-a680-10a9413b4a48 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:10.288671 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42509091-e801-490b-b753-8ce1271d7bc2 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:10.319454 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58117e-a276-483d-a680-10a9413b4a48 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:10.330406 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c766025-7054-4bbe-8d17-191bf99443cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:10.331141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c766025-7054-4bbe-8d17-191bf99443cc admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:10.331319 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c766025-7054-4bbe-8d17-191bf99443cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:10.331597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c766025-7054-4bbe-8d17-191bf99443cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:10.333138 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c766025-7054-4bbe-8d17-191bf99443cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:10.335166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 851/1757] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:09 2022] POST /identity/v3/users => generated 303 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:10.339522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c766025-7054-4bbe-8d17-191bf99443cc admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:10.343035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42509091-e801-490b-b753-8ce1271d7bc2 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:10.366132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 907/1758] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:10.367951 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42509091-e801-490b-b753-8ce1271d7bc2 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:10.373219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2959dcb-e9c6-4103-94b8-04e56d0989cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:10.374106 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2959dcb-e9c6-4103-94b8-04e56d0989cc admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:10.374592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2959dcb-e9c6-4103-94b8-04e56d0989cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:10.374710 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c766025-7054-4bbe-8d17-191bf99443cc admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:10.375173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2959dcb-e9c6-4103-94b8-04e56d0989cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:10.375827 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2959dcb-e9c6-4103-94b8-04e56d0989cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:10.377552 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2959dcb-e9c6-4103-94b8-04e56d0989cc admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:10.395679 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c766025-7054-4bbe-8d17-191bf99443cc admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:10.404304 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 852/1759] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:10.405233 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2959dcb-e9c6-4103-94b8-04e56d0989cc admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:10.409969 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d2959dcb-e9c6-4103-94b8-04e56d0989cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:10.411098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c4608-9904-4643-8f84-b44bf8a7f884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:10.412785 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c4608-9904-4643-8f84-b44bf8a7f884 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:10.413079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c4608-9904-4643-8f84-b44bf8a7f884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:10.413627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c4608-9904-4643-8f84-b44bf8a7f884 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:10.414312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c4608-9904-4643-8f84-b44bf8a7f884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:10.415986 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c4608-9904-4643-8f84-b44bf8a7f884 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:10.433195 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:10.466119 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 908/1760] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:10.467342 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c4608-9904-4643-8f84-b44bf8a7f884 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:10.490969 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c4608-9904-4643-8f84-b44bf8a7f884 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:10.512162 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8567fa6-43f8-4779-92aa-ddbf6a206ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:10.512802 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8567fa6-43f8-4779-92aa-ddbf6a206ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:10.513081 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8567fa6-43f8-4779-92aa-ddbf6a206ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:10.513508 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8567fa6-43f8-4779-92aa-ddbf6a206ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:10.514174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8567fa6-43f8-4779-92aa-ddbf6a206ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:10.516210 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8567fa6-43f8-4779-92aa-ddbf6a206ba1 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:10.532926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2959dcb-e9c6-4103-94b8-04e56d0989cc admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:10.538223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b8ff1-9f0c-4305-a6d8-504adcbf2d41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:10.538990 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b8ff1-9f0c-4305-a6d8-504adcbf2d41 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:10.539278 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b8ff1-9f0c-4305-a6d8-504adcbf2d41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:10.539662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b8ff1-9f0c-4305-a6d8-504adcbf2d41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:10.541638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2a7b8ff1-9f0c-4305-a6d8-504adcbf2d41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:10.545652 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 909/1761] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:10.546588 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8567fa6-43f8-4779-92aa-ddbf6a206ba1 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:10.569579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8567fa6-43f8-4779-92aa-ddbf6a206ba1 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:10.582255 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f6058-579d-433a-a818-199904686ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:10.584051 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f6058-579d-433a-a818-199904686ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:10.584347 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f6058-579d-433a-a818-199904686ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:10.584627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f6058-579d-433a-a818-199904686ea3 admin admin] PATH_INFO: `/v3/projects/9edf8e6e05cc49f386e6a77afd7c9f08/users/5ed4881a961f4cc596a9fa9d4d13dc0b/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:10.624494 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f6058-579d-433a-a818-199904686ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9edf8e6e05cc49f386e6a77afd7c9f08, user_id=5ed4881a961f4cc596a9fa9d4d13dc0b, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:10.629556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f6058-579d-433a-a818-199904686ea3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:10.663992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 910/1762] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:10 2022] PUT /identity/v3/projects/9edf8e6e05cc49f386e6a77afd7c9f08/users/5ed4881a961f4cc596a9fa9d4d13dc0b/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:10.663992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f6058-579d-433a-a818-199904686ea3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:10.712224 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b8ff1-9f0c-4305-a6d8-504adcbf2d41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:10.714043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b8ff1-9f0c-4305-a6d8-504adcbf2d41 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:10.726939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 853/1763] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:10.728774 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea5731-c28a-4eba-87cf-4f51aeba2f3c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:10.729232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea5731-c28a-4eba-87cf-4f51aeba2f3c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:10.729632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea5731-c28a-4eba-87cf-4f51aeba2f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:10.807127 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-bcea5731-c28a-4eba-87cf-4f51aeba2f3c None None] MFA Rules not processed for user `5ed4881a961f4cc596a9fa9d4d13dc0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:10.808824 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bcea5731-c28a-4eba-87cf-4f51aeba2f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:10.835348 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f6058-579d-433a-a818-199904686ea3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:10.841184 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aeaae4-5ae7-41a6-a9f4-5a8e24e99d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:10.842183 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeaae4-5ae7-41a6-a9f4-5a8e24e99d5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:10.842786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeaae4-5ae7-41a6-a9f4-5a8e24e99d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:10.843246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aeaae4-5ae7-41a6-a9f4-5a8e24e99d5d admin admin] PATH_INFO: `/v3/projects/76d859d5b38c418085ebac0d22fefe67/users/96b5bafdce6b4c768cfcfea71cdca6d5/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:10.864608 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aeaae4-5ae7-41a6-a9f4-5a8e24e99d5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d859d5b38c418085ebac0d22fefe67, user_id=96b5bafdce6b4c768cfcfea71cdca6d5, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:10.868291 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aeaae4-5ae7-41a6-a9f4-5a8e24e99d5d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:10.879050 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:10.884968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 911/1764] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:10 2022] PUT /identity/v3/projects/76d859d5b38c418085ebac0d22fefe67/users/96b5bafdce6b4c768cfcfea71cdca6d5/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:10.893158 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b80967-7320-435d-a4fd-04f811b33e59 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:10.893622 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b80967-7320-435d-a4fd-04f811b33e59 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:10.894522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b80967-7320-435d-a4fd-04f811b33e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:10.916428 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b1b80967-7320-435d-a4fd-04f811b33e59 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:10.917690 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b1b80967-7320-435d-a4fd-04f811b33e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:10.921786 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 854/1765] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:10.926472 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea5731-c28a-4eba-87cf-4f51aeba2f3c None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:10.938217 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea5731-c28a-4eba-87cf-4f51aeba2f3c None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:10.996374 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 912/1766] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:11.000893 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:11.007457 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b80967-7320-435d-a4fd-04f811b33e59 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:11.008805 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d259a-79d9-4089-9eb0-15425a59d937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:11.009212 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b1b80967-7320-435d-a4fd-04f811b33e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:11.009642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d259a-79d9-4089-9eb0-15425a59d937 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:11.009971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d259a-79d9-4089-9eb0-15425a59d937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:11.010355 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473d259a-79d9-4089-9eb0-15425a59d937 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:11.010997 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d259a-79d9-4089-9eb0-15425a59d937 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:11.013000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473d259a-79d9-4089-9eb0-15425a59d937 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:11.045499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 855/1767] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:10 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:11.050321 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d259a-79d9-4089-9eb0-15425a59d937 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:11.095808 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b80967-7320-435d-a4fd-04f811b33e59 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:11.152340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473d259a-79d9-4089-9eb0-15425a59d937 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:11.159187 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c02704-4353-45aa-afa0-d5a20af541e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:11.159817 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c02704-4353-45aa-afa0-d5a20af541e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:11.160152 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c02704-4353-45aa-afa0-d5a20af541e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:11.160553 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c02704-4353-45aa-afa0-d5a20af541e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:11.161581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c02704-4353-45aa-afa0-d5a20af541e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:11.165280 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c02704-4353-45aa-afa0-d5a20af541e4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:11.177011 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791d3f8-b41a-4160-a4e4-83412e04b31f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:11.178602 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791d3f8-b41a-4160-a4e4-83412e04b31f service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:11.179239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791d3f8-b41a-4160-a4e4-83412e04b31f service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:11.179754 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791d3f8-b41a-4160-a4e4-83412e04b31f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:11.196632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 856/1768] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:11 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:11.200991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c02704-4353-45aa-afa0-d5a20af541e4 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:11.205445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791d3f8-b41a-4160-a4e4-83412e04b31f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:11.207108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791d3f8-b41a-4160-a4e4-83412e04b31f service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:11.215390 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c02704-4353-45aa-afa0-d5a20af541e4 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:11.221850 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f825066-ccd9-40e3-a940-55df5d1f712f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:11.222475 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f825066-ccd9-40e3-a940-55df5d1f712f admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:11.222757 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f825066-ccd9-40e3-a940-55df5d1f712f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:11.223033 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f825066-ccd9-40e3-a940-55df5d1f712f admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:11.223827 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f825066-ccd9-40e3-a940-55df5d1f712f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:11.226755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f825066-ccd9-40e3-a940-55df5d1f712f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:11.232050 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 913/1769] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:11.243237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 857/1770] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:11.244555 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791d3f8-b41a-4160-a4e4-83412e04b31f service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:11.258177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791d3f8-b41a-4160-a4e4-83412e04b31f service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:11.267241 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa07231-1d7e-4148-805d-5ebac16458f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:11.268016 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa07231-1d7e-4148-805d-5ebac16458f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:11.268285 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa07231-1d7e-4148-805d-5ebac16458f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:11.268627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa07231-1d7e-4148-805d-5ebac16458f5 admin admin] PATH_INFO: `/v3/projects/2fd6925eae174e54ae1ac0464ba14f51/users/d608c3b8f285426e8d1aa34a98338d6f/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:11.318357 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa07231-1d7e-4148-805d-5ebac16458f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd6925eae174e54ae1ac0464ba14f51, user_id=d608c3b8f285426e8d1aa34a98338d6f, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:11.326561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa07231-1d7e-4148-805d-5ebac16458f5 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:11.364989 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 914/1771] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:11 2022] PUT /identity/v3/projects/2fd6925eae174e54ae1ac0464ba14f51/users/d608c3b8f285426e8d1aa34a98338d6f/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:11.369557 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f825066-ccd9-40e3-a940-55df5d1f712f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:11.385277 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f825066-ccd9-40e3-a940-55df5d1f712f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:11.497481 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695d14f-5e55-4d92-abfd-11d0bc908be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:11.500806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695d14f-5e55-4d92-abfd-11d0bc908be2 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:11.501796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695d14f-5e55-4d92-abfd-11d0bc908be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:11.502504 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695d14f-5e55-4d92-abfd-11d0bc908be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:11.504417 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695d14f-5e55-4d92-abfd-11d0bc908be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:11.513180 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695d14f-5e55-4d92-abfd-11d0bc908be2 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:11.538218 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa07231-1d7e-4148-805d-5ebac16458f5 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:11.541149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 858/1772] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:11.565320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa07231-1d7e-4148-805d-5ebac16458f5 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:11.574093 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef942b4c-1e1a-4f85-996b-e6c523178ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:11.575168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef942b4c-1e1a-4f85-996b-e6c523178ce2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:11.575626 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef942b4c-1e1a-4f85-996b-e6c523178ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:11.576113 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef942b4c-1e1a-4f85-996b-e6c523178ce2 admin admin] PATH_INFO: `/v3/projects/2fd6925eae174e54ae1ac0464ba14f51/users/d608c3b8f285426e8d1aa34a98338d6f/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:11.585887 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef942b4c-1e1a-4f85-996b-e6c523178ce2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd6925eae174e54ae1ac0464ba14f51, user_id=d608c3b8f285426e8d1aa34a98338d6f, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:11.590641 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef942b4c-1e1a-4f85-996b-e6c523178ce2 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:11.608123 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 915/1773] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:11 2022] PUT /identity/v3/projects/2fd6925eae174e54ae1ac0464ba14f51/users/d608c3b8f285426e8d1aa34a98338d6f/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:11.615072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d68caf-dcc8-4977-b3d9-c47cc851d884 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:11.615406 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d68caf-dcc8-4977-b3d9-c47cc851d884 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:11.615693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d68caf-dcc8-4977-b3d9-c47cc851d884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:11.650069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e6d68caf-dcc8-4977-b3d9-c47cc851d884 None None] MFA Rules not processed for user `d608c3b8f285426e8d1aa34a98338d6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:11.651863 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e6d68caf-dcc8-4977-b3d9-c47cc851d884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:11.703971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:11.742944 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 859/1774] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:11.751759 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef942b4c-1e1a-4f85-996b-e6c523178ce2 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:11.797492 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:11.834755 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef942b4c-1e1a-4f85-996b-e6c523178ce2 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:11.846487 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4547ec-3cc6-4e69-bded-a36eb3b2066d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:11.847402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4547ec-3cc6-4e69-bded-a36eb3b2066d service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:11.847509 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4547ec-3cc6-4e69-bded-a36eb3b2066d service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:11.848408 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4547ec-3cc6-4e69-bded-a36eb3b2066d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:11.870770 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4547ec-3cc6-4e69-bded-a36eb3b2066d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:11.872499 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4547ec-3cc6-4e69-bded-a36eb3b2066d service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:11.885546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 916/1775] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:11.994850 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d68caf-dcc8-4977-b3d9-c47cc851d884 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:12.006811 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d68caf-dcc8-4977-b3d9-c47cc851d884 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:12.010633 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334f83f-23ee-4d8d-81b8-057f1ffd7eaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:12.011253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334f83f-23ee-4d8d-81b8-057f1ffd7eaf service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:12.011534 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334f83f-23ee-4d8d-81b8-057f1ffd7eaf service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:12.011809 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334f83f-23ee-4d8d-81b8-057f1ffd7eaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:12.072871 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334f83f-23ee-4d8d-81b8-057f1ffd7eaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:12.074339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334f83f-23ee-4d8d-81b8-057f1ffd7eaf service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:12.083992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 860/1776] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:12.178606 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2087801-d61d-4267-8b4c-c9ecde29a312 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:12.179048 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2087801-d61d-4267-8b4c-c9ecde29a312 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:12.179779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2087801-d61d-4267-8b4c-c9ecde29a312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:12.230377 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f2087801-d61d-4267-8b4c-c9ecde29a312 None None] MFA Rules not processed for user `96b5bafdce6b4c768cfcfea71cdca6d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:12.232459 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f2087801-d61d-4267-8b4c-c9ecde29a312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:12.355898 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 917/1777] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:12 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:12.357953 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334f83f-23ee-4d8d-81b8-057f1ffd7eaf service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:12.459787 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334f83f-23ee-4d8d-81b8-057f1ffd7eaf service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:12.465184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de710c4-4573-4d21-ad5f-f2dfa26cf7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:12.465807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de710c4-4573-4d21-ad5f-f2dfa26cf7f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:12.466084 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de710c4-4573-4d21-ad5f-f2dfa26cf7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:12.466351 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de710c4-4573-4d21-ad5f-f2dfa26cf7f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:12.466976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de710c4-4573-4d21-ad5f-f2dfa26cf7f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:12.468936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de710c4-4573-4d21-ad5f-f2dfa26cf7f6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:12.500084 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 861/1778] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:12 2022] POST /identity/v3/projects => generated 405 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:12.502465 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2087801-d61d-4267-8b4c-c9ecde29a312 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:12.514098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2087801-d61d-4267-8b4c-c9ecde29a312 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:12.574977 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d308e17-a383-41fe-9631-4b951c78609d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:12.575699 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d308e17-a383-41fe-9631-4b951c78609d admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:12.576104 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d308e17-a383-41fe-9631-4b951c78609d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:12.576466 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d308e17-a383-41fe-9631-4b951c78609d admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:12.577172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d308e17-a383-41fe-9631-4b951c78609d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:12.579200 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d308e17-a383-41fe-9631-4b951c78609d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:12.608149 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 918/1779] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:12 2022] POST /identity/v3/users => generated 309 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:12.610243 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de710c4-4573-4d21-ad5f-f2dfa26cf7f6 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:12.621754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de710c4-4573-4d21-ad5f-f2dfa26cf7f6 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:12.626931 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352ac51-4865-44cf-babb-63a0a5926806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:12.627571 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352ac51-4865-44cf-babb-63a0a5926806 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:12.627886 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352ac51-4865-44cf-babb-63a0a5926806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:12.628189 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352ac51-4865-44cf-babb-63a0a5926806 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:12.628834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352ac51-4865-44cf-babb-63a0a5926806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:12.630428 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352ac51-4865-44cf-babb-63a0a5926806 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:12.641376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 862/1780] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:12.646302 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d308e17-a383-41fe-9631-4b951c78609d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:12.656723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d308e17-a383-41fe-9631-4b951c78609d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:12.662168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d56d4-abfa-4b40-8e17-30230fafd2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:12.662869 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d56d4-abfa-4b40-8e17-30230fafd2c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:12.663154 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d56d4-abfa-4b40-8e17-30230fafd2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:12.663431 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d56d4-abfa-4b40-8e17-30230fafd2c2 admin admin] PATH_INFO: `/v3/projects/d0d0d0e54cf3417588444bc312a2aa06/users/08106dd4ab07452a9c1ee886a3d3112c/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:12.680693 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9d56d4-abfa-4b40-8e17-30230fafd2c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0d0d0e54cf3417588444bc312a2aa06, user_id=08106dd4ab07452a9c1ee886a3d3112c, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:12.683423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9d56d4-abfa-4b40-8e17-30230fafd2c2 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:12.696520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 919/1781] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:12 2022] PUT /identity/v3/projects/d0d0d0e54cf3417588444bc312a2aa06/users/08106dd4ab07452a9c1ee886a3d3112c/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:12.700556 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352ac51-4865-44cf-babb-63a0a5926806 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:12.711192 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352ac51-4865-44cf-babb-63a0a5926806 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:12.766061 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:12.776771 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d58f3f-251d-4046-a482-354cfc2fcd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:12.777404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d58f3f-251d-4046-a482-354cfc2fcd19 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:12.777682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d58f3f-251d-4046-a482-354cfc2fcd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:12.777953 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d58f3f-251d-4046-a482-354cfc2fcd19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:12.778591 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d58f3f-251d-4046-a482-354cfc2fcd19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:12.780252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d58f3f-251d-4046-a482-354cfc2fcd19 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:12.791052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 863/1782] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:12.796967 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d56d4-abfa-4b40-8e17-30230fafd2c2 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:12.807855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d56d4-abfa-4b40-8e17-30230fafd2c2 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:12.812560 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f26fda-ab19-4508-8dee-628c68c3d636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:12.813224 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f26fda-ab19-4508-8dee-628c68c3d636 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:12.813523 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f26fda-ab19-4508-8dee-628c68c3d636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:12.813784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f26fda-ab19-4508-8dee-628c68c3d636 admin admin] PATH_INFO: `/v3/projects/d0d0d0e54cf3417588444bc312a2aa06/users/08106dd4ab07452a9c1ee886a3d3112c/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:12.815670 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f26fda-ab19-4508-8dee-628c68c3d636 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0d0d0e54cf3417588444bc312a2aa06, user_id=08106dd4ab07452a9c1ee886a3d3112c, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:12.818345 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f26fda-ab19-4508-8dee-628c68c3d636 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:12.829014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 920/1783] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:12 2022] PUT /identity/v3/projects/d0d0d0e54cf3417588444bc312a2aa06/users/08106dd4ab07452a9c1ee886a3d3112c/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:13.728064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a0d03-1968-449e-8aa3-d27faca75037 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:13.728619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a0d03-1968-449e-8aa3-d27faca75037 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:13.728619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a0d03-1968-449e-8aa3-d27faca75037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:13.760073 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-299a0d03-1968-449e-8aa3-d27faca75037 None None] MFA Rules not processed for user `08106dd4ab07452a9c1ee886a3d3112c`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:13.761312 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-299a0d03-1968-449e-8aa3-d27faca75037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:13.868488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 864/1784] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:13 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:14.805645 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017e6a8-c985-4749-b571-240aade2ed56 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:14.805645 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017e6a8-c985-4749-b571-240aade2ed56 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:14.806341 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017e6a8-c985-4749-b571-240aade2ed56 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:14.807172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 921/1785] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 29 20:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:33:14.840032 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd4c15-1fd3-44f3-8010-4a1a85a47c93 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:14.840032 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd4c15-1fd3-44f3-8010-4a1a85a47c93 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:14.840714 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd4c15-1fd3-44f3-8010-4a1a85a47c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:14.864642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-21dd4c15-1fd3-44f3-8010-4a1a85a47c93 None None] MFA Rules not processed for user `fa0e659eee5e4a9da8d3d7337cb73798`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:14.867022 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-21dd4c15-1fd3-44f3-8010-4a1a85a47c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:15.044171 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 865/1786] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 29 20:33:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:15.074939 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017e6a8-c985-4749-b571-240aade2ed56 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:15.076947 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3017e6a8-c985-4749-b571-240aade2ed56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:15.121405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3017e6a8-c985-4749-b571-240aade2ed56 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:15.138181 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d5824-e148-44d1-a7c9-e9c0a00cfec9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0e659eee5e4a9da8d3d7337cb73798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:15.140465 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d5824-e148-44d1-a7c9-e9c0a00cfec9 service placement] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:15.141912 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d5824-e148-44d1-a7c9-e9c0a00cfec9 service placement] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:15.145557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d5824-e148-44d1-a7c9-e9c0a00cfec9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:15.167385 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:15.195451 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d5824-e148-44d1-a7c9-e9c0a00cfec9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:15.202887 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d5824-e148-44d1-a7c9-e9c0a00cfec9 service placement] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:15.240304 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 922/1787] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:16.256477 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd4c15-1fd3-44f3-8010-4a1a85a47c93 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:16.330295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd4c15-1fd3-44f3-8010-4a1a85a47c93 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:16.335108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e9b4b-3f76-4a7d-b3df-1d304163d7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:16.335840 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e9b4b-3f76-4a7d-b3df-1d304163d7a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:16.336162 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e9b4b-3f76-4a7d-b3df-1d304163d7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:16.336432 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e9b4b-3f76-4a7d-b3df-1d304163d7a6 admin admin] PATH_INFO: `/v3/users/96b5bafdce6b4c768cfcfea71cdca6d5` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:16.337689 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e9b4b-3f76-4a7d-b3df-1d304163d7a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96b5bafdce6b4c768cfcfea71cdca6d5)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:16.339682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e9b4b-3f76-4a7d-b3df-1d304163d7a6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:16.357662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:16.416450 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-369e9b4b-3f76-4a7d-b3df-1d304163d7a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96b5bafdce6b4c768cfcfea71cdca6d5'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:16.430144 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-369e9b4b-3f76-4a7d-b3df-1d304163d7a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96b5bafdce6b4c768cfcfea71cdca6d5'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:16.440139 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-369e9b4b-3f76-4a7d-b3df-1d304163d7a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96b5bafdce6b4c768cfcfea71cdca6d5'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:16.441001 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-369e9b4b-3f76-4a7d-b3df-1d304163d7a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96b5bafdce6b4c768cfcfea71cdca6d5'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:16.457283 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-369e9b4b-3f76-4a7d-b3df-1d304163d7a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96b5bafdce6b4c768cfcfea71cdca6d5'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:16.465538 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 866/1788] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:16 2022] DELETE /identity/v3/users/96b5bafdce6b4c768cfcfea71cdca6d5 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:16.722870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d5824-e148-44d1-a7c9-e9c0a00cfec9 service placement] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:16.738116 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d5824-e148-44d1-a7c9-e9c0a00cfec9 service placement] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:16.831318 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d2a69-8faa-4829-a0ee-ac3337fe3250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:16.832665 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d2a69-8faa-4829-a0ee-ac3337fe3250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:16.832952 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d2a69-8faa-4829-a0ee-ac3337fe3250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:16.833236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d2a69-8faa-4829-a0ee-ac3337fe3250 admin admin] PATH_INFO: `/v3/projects/76d859d5b38c418085ebac0d22fefe67` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:16.834423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d2a69-8faa-4829-a0ee-ac3337fe3250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d859d5b38c418085ebac0d22fefe67)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:16.836454 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3d2a69-8faa-4829-a0ee-ac3337fe3250 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:16.916207 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8c3d2a69-8faa-4829-a0ee-ac3337fe3250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d859d5b38c418085ebac0d22fefe67', 'request_id': 'req-8c3d2a69-8faa-4829-a0ee-ac3337fe3250'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:16.930113 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8c3d2a69-8faa-4829-a0ee-ac3337fe3250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d859d5b38c418085ebac0d22fefe67', 'request_id': 'req-8c3d2a69-8faa-4829-a0ee-ac3337fe3250'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:16.937753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8c3d2a69-8faa-4829-a0ee-ac3337fe3250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d859d5b38c418085ebac0d22fefe67', 'request_id': 'req-8c3d2a69-8faa-4829-a0ee-ac3337fe3250'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:16.951310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8c3d2a69-8faa-4829-a0ee-ac3337fe3250 admin admin] The token cache is being invalidate because project 76d859d5b38c418085ebac0d22fefe67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:16.951677 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8c3d2a69-8faa-4829-a0ee-ac3337fe3250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:16.955975 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8c3d2a69-8faa-4829-a0ee-ac3337fe3250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:16.958033 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 923/1789] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:16 2022] DELETE /identity/v3/projects/76d859d5b38c418085ebac0d22fefe67 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:16.961990 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e9b4b-3f76-4a7d-b3df-1d304163d7a6 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:16.963502 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-369e9b4b-3f76-4a7d-b3df-1d304163d7a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:16.978261 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e9b4b-3f76-4a7d-b3df-1d304163d7a6 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:17.051453 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff59155-7789-4a3e-a48c-923ed14a4587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:17.052288 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff59155-7789-4a3e-a48c-923ed14a4587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:17.052660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff59155-7789-4a3e-a48c-923ed14a4587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:17.052776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff59155-7789-4a3e-a48c-923ed14a4587 admin admin] PATH_INFO: `/v3/users/08106dd4ab07452a9c1ee886a3d3112c` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:17.053945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff59155-7789-4a3e-a48c-923ed14a4587 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08106dd4ab07452a9c1ee886a3d3112c)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:17.055875 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff59155-7789-4a3e-a48c-923ed14a4587 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:17.108648 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8ff59155-7789-4a3e-a48c-923ed14a4587 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08106dd4ab07452a9c1ee886a3d3112c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:17.115886 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8ff59155-7789-4a3e-a48c-923ed14a4587 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08106dd4ab07452a9c1ee886a3d3112c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:17.123894 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8ff59155-7789-4a3e-a48c-923ed14a4587 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08106dd4ab07452a9c1ee886a3d3112c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:17.124599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8ff59155-7789-4a3e-a48c-923ed14a4587 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08106dd4ab07452a9c1ee886a3d3112c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:17.136908 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8ff59155-7789-4a3e-a48c-923ed14a4587 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08106dd4ab07452a9c1ee886a3d3112c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:17.143698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 867/1790] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:16 2022] DELETE /identity/v3/users/08106dd4ab07452a9c1ee886a3d3112c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:17.349950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d2a69-8faa-4829-a0ee-ac3337fe3250 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:17.351442 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-8c3d2a69-8faa-4829-a0ee-ac3337fe3250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:17.364713 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d2a69-8faa-4829-a0ee-ac3337fe3250 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:17.432167 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9644a68-bb1c-425b-842d-d471bac78d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:17.432830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9644a68-bb1c-425b-842d-d471bac78d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:17.433116 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9644a68-bb1c-425b-842d-d471bac78d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:17.433392 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9644a68-bb1c-425b-842d-d471bac78d6a admin admin] PATH_INFO: `/v3/projects/d0d0d0e54cf3417588444bc312a2aa06` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:17.435231 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9644a68-bb1c-425b-842d-d471bac78d6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0d0d0e54cf3417588444bc312a2aa06)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:17.437643 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9644a68-bb1c-425b-842d-d471bac78d6a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:17.479959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:17.502800 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e9644a68-bb1c-425b-842d-d471bac78d6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0d0d0e54cf3417588444bc312a2aa06', 'request_id': 'req-e9644a68-bb1c-425b-842d-d471bac78d6a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:17.511859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e9644a68-bb1c-425b-842d-d471bac78d6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0d0d0e54cf3417588444bc312a2aa06', 'request_id': 'req-e9644a68-bb1c-425b-842d-d471bac78d6a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:17.520354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e9644a68-bb1c-425b-842d-d471bac78d6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0d0d0e54cf3417588444bc312a2aa06', 'request_id': 'req-e9644a68-bb1c-425b-842d-d471bac78d6a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:17.533403 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e9644a68-bb1c-425b-842d-d471bac78d6a admin admin] The token cache is being invalidate because project d0d0d0e54cf3417588444bc312a2aa06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:17.533700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e9644a68-bb1c-425b-842d-d471bac78d6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:17.534464 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e9644a68-bb1c-425b-842d-d471bac78d6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:17.536130 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 924/1791] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:17 2022] DELETE /identity/v3/projects/d0d0d0e54cf3417588444bc312a2aa06 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:17.541021 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de239d53-4f55-44c5-9602-6115b0e16b5e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:17.541529 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de239d53-4f55-44c5-9602-6115b0e16b5e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:17.541611 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de239d53-4f55-44c5-9602-6115b0e16b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:17.564299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-de239d53-4f55-44c5-9602-6115b0e16b5e None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:17.566191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-de239d53-4f55-44c5-9602-6115b0e16b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:17.571533 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:17.646255 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 868/1792] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:17.656944 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8d3ed-341d-4d7a-b330-ad51740a78f0 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:17.657421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8d3ed-341d-4d7a-b330-ad51740a78f0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:17.657893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8d3ed-341d-4d7a-b330-ad51740a78f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:17.683193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-dce8d3ed-341d-4d7a-b330-ad51740a78f0 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:17.684455 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-dce8d3ed-341d-4d7a-b330-ad51740a78f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:17.694576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 925/1793] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:17.699047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de239d53-4f55-44c5-9602-6115b0e16b5e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:17.711813 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de239d53-4f55-44c5-9602-6115b0e16b5e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:17.718073 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aba893-7f04-45a1-8ead-c15b4a15ad95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:17.718701 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aba893-7f04-45a1-8ead-c15b4a15ad95 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:17.719003 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aba893-7f04-45a1-8ead-c15b4a15ad95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:17.719262 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aba893-7f04-45a1-8ead-c15b4a15ad95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:17.720125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aba893-7f04-45a1-8ead-c15b4a15ad95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:33:17.720531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aba893-7f04-45a1-8ead-c15b4a15ad95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:17.722247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aba893-7f04-45a1-8ead-c15b4a15ad95 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:17.738063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 869/1794] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:33:17.742479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8d3ed-341d-4d7a-b330-ad51740a78f0 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:17.753465 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8d3ed-341d-4d7a-b330-ad51740a78f0 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:17.758174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56fdc4-3d82-468c-9898-7752d94a5a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:17.759188 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56fdc4-3d82-468c-9898-7752d94a5a10 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:17.759617 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56fdc4-3d82-468c-9898-7752d94a5a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:17.759902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56fdc4-3d82-468c-9898-7752d94a5a10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:17.760711 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56fdc4-3d82-468c-9898-7752d94a5a10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:17.763087 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56fdc4-3d82-468c-9898-7752d94a5a10 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:17.798177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 926/1795] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:17 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:17.799898 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aba893-7f04-45a1-8ead-c15b4a15ad95 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:17.811171 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aba893-7f04-45a1-8ead-c15b4a15ad95 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:17.866733 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38df4f-fa08-4488-8c7b-a7de2c5ddda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:17.867340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38df4f-fa08-4488-8c7b-a7de2c5ddda7 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:17.867620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38df4f-fa08-4488-8c7b-a7de2c5ddda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:17.867890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38df4f-fa08-4488-8c7b-a7de2c5ddda7 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:17.868560 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db38df4f-fa08-4488-8c7b-a7de2c5ddda7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:17.870499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db38df4f-fa08-4488-8c7b-a7de2c5ddda7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:17.894027 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 870/1796] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:17 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:17.898456 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56fdc4-3d82-468c-9898-7752d94a5a10 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:17.909940 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56fdc4-3d82-468c-9898-7752d94a5a10 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:17.919514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88560bd3-45ce-48bf-9568-9bcfb20b585a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:17.920205 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88560bd3-45ce-48bf-9568-9bcfb20b585a admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:17.920886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88560bd3-45ce-48bf-9568-9bcfb20b585a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:17.921277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88560bd3-45ce-48bf-9568-9bcfb20b585a admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:17.921834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88560bd3-45ce-48bf-9568-9bcfb20b585a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:17.923632 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88560bd3-45ce-48bf-9568-9bcfb20b585a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:17.938126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 927/1797] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:17.941158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38df4f-fa08-4488-8c7b-a7de2c5ddda7 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:17.951715 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38df4f-fa08-4488-8c7b-a7de2c5ddda7 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:17.956713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc7d8f-aa10-4369-a569-b5afc2606f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:17.957337 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc7d8f-aa10-4369-a569-b5afc2606f91 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:17.957615 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc7d8f-aa10-4369-a569-b5afc2606f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:17.957880 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc7d8f-aa10-4369-a569-b5afc2606f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:17.958536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc7d8f-aa10-4369-a569-b5afc2606f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:17.961287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc7d8f-aa10-4369-a569-b5afc2606f91 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:17.975691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 871/1798] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:17.980199 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88560bd3-45ce-48bf-9568-9bcfb20b585a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:17.990868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88560bd3-45ce-48bf-9568-9bcfb20b585a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:17.995756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ca376-61f9-42e6-8312-fd0971c23c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:17.997004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ca376-61f9-42e6-8312-fd0971c23c37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:17.997004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ca376-61f9-42e6-8312-fd0971c23c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:17.997004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ca376-61f9-42e6-8312-fd0971c23c37 admin admin] PATH_INFO: `/v3/projects/8c20e3253ec6494886386d64e7883e61/users/e8da4e9c0fc2411bb44611afaba8cbc2/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:18.014256 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ca376-61f9-42e6-8312-fd0971c23c37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c20e3253ec6494886386d64e7883e61, user_id=e8da4e9c0fc2411bb44611afaba8cbc2, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:18.017238 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ca376-61f9-42e6-8312-fd0971c23c37 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:18.032859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 928/1799] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:17 2022] PUT /identity/v3/projects/8c20e3253ec6494886386d64e7883e61/users/e8da4e9c0fc2411bb44611afaba8cbc2/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:18.044665 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23c3df-1c13-4ce9-ac3e-3f0015c032db None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:18.044948 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23c3df-1c13-4ce9-ac3e-3f0015c032db None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:18.045272 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23c3df-1c13-4ce9-ac3e-3f0015c032db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:18.080404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5b23c3df-1c13-4ce9-ac3e-3f0015c032db None None] MFA Rules not processed for user `e8da4e9c0fc2411bb44611afaba8cbc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:18.081622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5b23c3df-1c13-4ce9-ac3e-3f0015c032db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:18.156247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 872/1800] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:18.161491 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ca376-61f9-42e6-8312-fd0971c23c37 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:18.232131 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ca376-61f9-42e6-8312-fd0971c23c37 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:18.237465 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177be8f-c50c-48ee-a570-afe7230eb047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:18.238072 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177be8f-c50c-48ee-a570-afe7230eb047 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:18.238383 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177be8f-c50c-48ee-a570-afe7230eb047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:18.238660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e177be8f-c50c-48ee-a570-afe7230eb047 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:18.239310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177be8f-c50c-48ee-a570-afe7230eb047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:18.241250 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e177be8f-c50c-48ee-a570-afe7230eb047 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:18.268747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 929/1801] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:18 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:18.273143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23c3df-1c13-4ce9-ac3e-3f0015c032db None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:18.281506 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:18.298153 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23c3df-1c13-4ce9-ac3e-3f0015c032db None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:18.367799 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bbbc3-a9b6-4ae4-b911-0d91d03147cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:18.368446 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bbbc3-a9b6-4ae4-b911-0d91d03147cd admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:18.368720 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bbbc3-a9b6-4ae4-b911-0d91d03147cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:18.368988 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bbbc3-a9b6-4ae4-b911-0d91d03147cd admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:18.369652 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bbbc3-a9b6-4ae4-b911-0d91d03147cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:18.371600 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bbbc3-a9b6-4ae4-b911-0d91d03147cd admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:18.396081 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 873/1802] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:18 2022] POST /identity/v3/users => generated 292 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:18.399794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177be8f-c50c-48ee-a570-afe7230eb047 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:18.413588 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e177be8f-c50c-48ee-a570-afe7230eb047 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:18.418961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614810e7-2c71-4a1d-a5df-c7b6fcb0da1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:18.419584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614810e7-2c71-4a1d-a5df-c7b6fcb0da1e admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:18.419869 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614810e7-2c71-4a1d-a5df-c7b6fcb0da1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:18.420177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614810e7-2c71-4a1d-a5df-c7b6fcb0da1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:18.421087 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614810e7-2c71-4a1d-a5df-c7b6fcb0da1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:18.422687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614810e7-2c71-4a1d-a5df-c7b6fcb0da1e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:18.435143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 930/1803] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:18.439633 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bbbc3-a9b6-4ae4-b911-0d91d03147cd admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:18.449510 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bbbc3-a9b6-4ae4-b911-0d91d03147cd admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:18.455434 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1d035-db4a-45d2-a199-efa825bda8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:18.456125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1d035-db4a-45d2-a199-efa825bda8df admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:18.456404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1d035-db4a-45d2-a199-efa825bda8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:18.456680 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1d035-db4a-45d2-a199-efa825bda8df admin admin] PATH_INFO: `/v3/projects/c1a336ba0d2c46cb91eb774adb7ef471/users/60a43d653a234f96bf4b9bdd250d15a8/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:18.473002 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1d035-db4a-45d2-a199-efa825bda8df admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1a336ba0d2c46cb91eb774adb7ef471, user_id=60a43d653a234f96bf4b9bdd250d15a8, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:18.475786 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1d035-db4a-45d2-a199-efa825bda8df admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:18.492721 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 874/1804] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:18 2022] PUT /identity/v3/projects/c1a336ba0d2c46cb91eb774adb7ef471/users/60a43d653a234f96bf4b9bdd250d15a8/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:18.498093 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614810e7-2c71-4a1d-a5df-c7b6fcb0da1e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:18.512436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d95f35-c077-4004-b277-00bbce73f06d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:18.512727 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d95f35-c077-4004-b277-00bbce73f06d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:18.513056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d95f35-c077-4004-b277-00bbce73f06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:18.521777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614810e7-2c71-4a1d-a5df-c7b6fcb0da1e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:18.547503 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-76d95f35-c077-4004-b277-00bbce73f06d None None] MFA Rules not processed for user `d51e331a4c9c4f038bdb80f8c5b74e42`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:18.551640 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-76d95f35-c077-4004-b277-00bbce73f06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:18.565239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:18.620584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284a674-6037-49f1-8d62-8730d5b50dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:18.621245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284a674-6037-49f1-8d62-8730d5b50dcc admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:18.621527 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284a674-6037-49f1-8d62-8730d5b50dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:18.621958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284a674-6037-49f1-8d62-8730d5b50dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:18.623219 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284a674-6037-49f1-8d62-8730d5b50dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:18.624563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284a674-6037-49f1-8d62-8730d5b50dcc admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:18.635817 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 875/1805] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Aug 29 20:33:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:18.638816 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 931/1806] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:18.644158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d95f35-c077-4004-b277-00bbce73f06d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:18.661016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d95f35-c077-4004-b277-00bbce73f06d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:18.665899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01b3cd-86f8-44c2-bf14-951a2e8999cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:18.666583 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01b3cd-86f8-44c2-bf14-951a2e8999cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:18.667024 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01b3cd-86f8-44c2-bf14-951a2e8999cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:18.667299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01b3cd-86f8-44c2-bf14-951a2e8999cd admin admin] PATH_INFO: `/v3/projects/c1a336ba0d2c46cb91eb774adb7ef471/users/60a43d653a234f96bf4b9bdd250d15a8/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:18.669864 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01b3cd-86f8-44c2-bf14-951a2e8999cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1a336ba0d2c46cb91eb774adb7ef471, user_id=60a43d653a234f96bf4b9bdd250d15a8, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:18.672884 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284a674-6037-49f1-8d62-8730d5b50dcc admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:18.674968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f284a674-6037-49f1-8d62-8730d5b50dcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:18.687336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01b3cd-86f8-44c2-bf14-951a2e8999cd admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:18.696178 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284a674-6037-49f1-8d62-8730d5b50dcc admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:18.708990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197440a-4636-47d4-aef3-5cad064044f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:18.709693 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197440a-4636-47d4-aef3-5cad064044f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:18.710388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197440a-4636-47d4-aef3-5cad064044f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:18.710732 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197440a-4636-47d4-aef3-5cad064044f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:18.719628 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 876/1807] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:18 2022] PUT /identity/v3/projects/c1a336ba0d2c46cb91eb774adb7ef471/users/60a43d653a234f96bf4b9bdd250d15a8/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:18.727010 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197440a-4636-47d4-aef3-5cad064044f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:18.729025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197440a-4636-47d4-aef3-5cad064044f7 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:18.741564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5adbc-7d2d-41fc-994b-4383b59dbb32 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:18.741941 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5adbc-7d2d-41fc-994b-4383b59dbb32 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:18.742322 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5adbc-7d2d-41fc-994b-4383b59dbb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:18.747730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 932/1808] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:33:18.793073 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6ec5adbc-7d2d-41fc-994b-4383b59dbb32 None None] MFA Rules not processed for user `60a43d653a234f96bf4b9bdd250d15a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:18.794550 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6ec5adbc-7d2d-41fc-994b-4383b59dbb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:18.814472 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f973f-5e7f-4e8f-82f6-be0d5044d1cc None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:18.815020 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f973f-5e7f-4e8f-82f6-be0d5044d1cc None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:18.815802 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f973f-5e7f-4e8f-82f6-be0d5044d1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:18.923680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-909f973f-5e7f-4e8f-82f6-be0d5044d1cc None None] MFA Rules not processed for user `61709ccc1e244185b252179451cd860f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:18.927866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:18.929821 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-909f973f-5e7f-4e8f-82f6-be0d5044d1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:18.961129 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 877/1809] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:18.970636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5adbc-7d2d-41fc-994b-4383b59dbb32 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:18.972728 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6ec5adbc-7d2d-41fc-994b-4383b59dbb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:19.083386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 933/1810] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 29 20:33:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:19.204409 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5adbc-7d2d-41fc-994b-4383b59dbb32 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:19.211981 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894f159-64b0-4d2c-b613-55259e3fedc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:19.213228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894f159-64b0-4d2c-b613-55259e3fedc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:19.213906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894f159-64b0-4d2c-b613-55259e3fedc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:19.214448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894f159-64b0-4d2c-b613-55259e3fedc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:19.229837 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894f159-64b0-4d2c-b613-55259e3fedc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:19.232229 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9894f159-64b0-4d2c-b613-55259e3fedc5 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:19.249590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 878/1811] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:19.302819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f973f-5e7f-4e8f-82f6-be0d5044d1cc None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:19.304377 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-909f973f-5e7f-4e8f-82f6-be0d5044d1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:19.487990 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894f159-64b0-4d2c-b613-55259e3fedc5 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:19.488368 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9894f159-64b0-4d2c-b613-55259e3fedc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:19.563812 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:19.591641 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f973f-5e7f-4e8f-82f6-be0d5044d1cc None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:19.607896 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd184bb-d1cc-47d3-9bc1-8ee26cce4cbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599129b7d8ff47efb145787414ddb80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:19.608697 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd184bb-d1cc-47d3-9bc1-8ee26cce4cbf service swift] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:19.609048 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd184bb-d1cc-47d3-9bc1-8ee26cce4cbf service swift] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:19.609336 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd184bb-d1cc-47d3-9bc1-8ee26cce4cbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:19.630083 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd184bb-d1cc-47d3-9bc1-8ee26cce4cbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:19.634028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd184bb-d1cc-47d3-9bc1-8ee26cce4cbf service swift] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:19.656226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 934/1812] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 29 20:33:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:33:19.696738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894f159-64b0-4d2c-b613-55259e3fedc5 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:19.706543 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc09b9a-7914-44e4-8964-2ef1c4bb4f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:19.707297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc09b9a-7914-44e4-8964-2ef1c4bb4f92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:19.707636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc09b9a-7914-44e4-8964-2ef1c4bb4f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:19.708062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc09b9a-7914-44e4-8964-2ef1c4bb4f92 admin admin] PATH_INFO: `/v3/users/5ed4881a961f4cc596a9fa9d4d13dc0b` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:19.709437 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc09b9a-7914-44e4-8964-2ef1c4bb4f92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ed4881a961f4cc596a9fa9d4d13dc0b)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:19.712273 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc09b9a-7914-44e4-8964-2ef1c4bb4f92 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:19.868828 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2bc09b9a-7914-44e4-8964-2ef1c4bb4f92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ed4881a961f4cc596a9fa9d4d13dc0b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:19.904421 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2bc09b9a-7914-44e4-8964-2ef1c4bb4f92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ed4881a961f4cc596a9fa9d4d13dc0b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:19.936377 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2bc09b9a-7914-44e4-8964-2ef1c4bb4f92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ed4881a961f4cc596a9fa9d4d13dc0b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:19.938672 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2bc09b9a-7914-44e4-8964-2ef1c4bb4f92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ed4881a961f4cc596a9fa9d4d13dc0b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:19.996531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2bc09b9a-7914-44e4-8964-2ef1c4bb4f92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ed4881a961f4cc596a9fa9d4d13dc0b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:20.014939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 879/1813] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:19 2022] DELETE /identity/v3/users/5ed4881a961f4cc596a9fa9d4d13dc0b => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:20.016567 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4eedf8-2500-44b2-9f04-abb3e4ce2e27 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:20.017077 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4eedf8-2500-44b2-9f04-abb3e4ce2e27 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:20.017269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4eedf8-2500-44b2-9f04-abb3e4ce2e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:20.069787 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc09b9a-7914-44e4-8964-2ef1c4bb4f92 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:20.073993 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2bc09b9a-7914-44e4-8964-2ef1c4bb4f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:20.118602 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-eb4eedf8-2500-44b2-9f04-abb3e4ce2e27 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:20.124425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-eb4eedf8-2500-44b2-9f04-abb3e4ce2e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:20.133985 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:20.329758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc09b9a-7914-44e4-8964-2ef1c4bb4f92 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:20.347897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedafbcf-cd9d-4a61-940e-1543b6e228ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:20.350412 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedafbcf-cd9d-4a61-940e-1543b6e228ee service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:20.351296 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedafbcf-cd9d-4a61-940e-1543b6e228ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:20.352580 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedafbcf-cd9d-4a61-940e-1543b6e228ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:20.356564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-fedafbcf-cd9d-4a61-940e-1543b6e228ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:20.366782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 935/1814] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:20.440285 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4eedf8-2500-44b2-9f04-abb3e4ce2e27 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:20.440285 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-eb4eedf8-2500-44b2-9f04-abb3e4ce2e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:20.541027 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedafbcf-cd9d-4a61-940e-1543b6e228ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:20.542574 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedafbcf-cd9d-4a61-940e-1543b6e228ee service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:20.568017 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 880/1815] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:33:20.629480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4eedf8-2500-44b2-9f04-abb3e4ce2e27 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:20.647101 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ddcce-77f4-4421-bdec-3b708170bbc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:20.647965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ddcce-77f4-4421-bdec-3b708170bbc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:20.648404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ddcce-77f4-4421-bdec-3b708170bbc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:20.648834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ddcce-77f4-4421-bdec-3b708170bbc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:20.671872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ddcce-77f4-4421-bdec-3b708170bbc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:20.673921 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ddcce-77f4-4421-bdec-3b708170bbc8 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:20.704371 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 936/1816] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:33:20.777541 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1340f58-71df-479f-9924-c26302665607 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:20.778092 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1340f58-71df-479f-9924-c26302665607 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:20.778312 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1340f58-71df-479f-9924-c26302665607 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:20.779828 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 881/1817] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 29 20:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:33:20.866183 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12e0b6-f441-456c-a4f6-46e8dbbc5632 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:20.866719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12e0b6-f441-456c-a4f6-46e8dbbc5632 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:20.867140 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12e0b6-f441-456c-a4f6-46e8dbbc5632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:20.902833 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1340f58-71df-479f-9924-c26302665607 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:20.905140 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f1340f58-71df-479f-9924-c26302665607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:20.938287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-7a12e0b6-f441-456c-a4f6-46e8dbbc5632 None None] MFA Rules not processed for user `c0c44754a1c04a2b87081aba6cb509d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:20.941886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7a12e0b6-f441-456c-a4f6-46e8dbbc5632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:20.953461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1340f58-71df-479f-9924-c26302665607 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:20.966183 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6066fcf-c7ac-4743-a62b-94512e8dd8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:20.966877 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6066fcf-c7ac-4743-a62b-94512e8dd8f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:20.967240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6066fcf-c7ac-4743-a62b-94512e8dd8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:20.967596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6066fcf-c7ac-4743-a62b-94512e8dd8f4 admin admin] PATH_INFO: `/v3/users/e8da4e9c0fc2411bb44611afaba8cbc2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:20.969219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6066fcf-c7ac-4743-a62b-94512e8dd8f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8da4e9c0fc2411bb44611afaba8cbc2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:20.973476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6066fcf-c7ac-4743-a62b-94512e8dd8f4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:21.088151 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e6066fcf-c7ac-4743-a62b-94512e8dd8f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8da4e9c0fc2411bb44611afaba8cbc2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:21.106216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e6066fcf-c7ac-4743-a62b-94512e8dd8f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8da4e9c0fc2411bb44611afaba8cbc2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:21.120587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e6066fcf-c7ac-4743-a62b-94512e8dd8f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8da4e9c0fc2411bb44611afaba8cbc2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:21.121292 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e6066fcf-c7ac-4743-a62b-94512e8dd8f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8da4e9c0fc2411bb44611afaba8cbc2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:21.151509 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e6066fcf-c7ac-4743-a62b-94512e8dd8f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8da4e9c0fc2411bb44611afaba8cbc2'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:21.180061 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 882/1818] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:20 2022] DELETE /identity/v3/users/e8da4e9c0fc2411bb44611afaba8cbc2 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:21.184444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b402a-6fc4-4efc-9097-b74d4ec60085 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:21.184748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b402a-6fc4-4efc-9097-b74d4ec60085 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:21.186770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b402a-6fc4-4efc-9097-b74d4ec60085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:21.242927 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-9c5b402a-6fc4-4efc-9097-b74d4ec60085 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:21.249366 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9c5b402a-6fc4-4efc-9097-b74d4ec60085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:21.359082 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:21.390198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 937/1819] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 29 20:33:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 523 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:21.429909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 883/1820] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:21.439822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12e0b6-f441-456c-a4f6-46e8dbbc5632 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:21.442464 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7a12e0b6-f441-456c-a4f6-46e8dbbc5632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:21.479567 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12e0b6-f441-456c-a4f6-46e8dbbc5632 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:21.503726 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b402a-6fc4-4efc-9097-b74d4ec60085 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:21.509207 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9c5b402a-6fc4-4efc-9097-b74d4ec60085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:21.646211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:21.687360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b402a-6fc4-4efc-9097-b74d4ec60085 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:21.695625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b65604-7ab3-4fdf-bf27-a1ad41813667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:21.696516 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b65604-7ab3-4fdf-bf27-a1ad41813667 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:21.696813 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b65604-7ab3-4fdf-bf27-a1ad41813667 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:21.697088 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b65604-7ab3-4fdf-bf27-a1ad41813667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:21.713273 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b65604-7ab3-4fdf-bf27-a1ad41813667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:21.726069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b65604-7ab3-4fdf-bf27-a1ad41813667 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:21.758043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 884/1821] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:21.778059 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b43b52-27ff-469a-b7d2-00971fbfd698 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:21.778721 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b43b52-27ff-469a-b7d2-00971fbfd698 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:21.779008 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b43b52-27ff-469a-b7d2-00971fbfd698 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:21.779292 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b43b52-27ff-469a-b7d2-00971fbfd698 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:21.806521 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b43b52-27ff-469a-b7d2-00971fbfd698 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:21.830490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b43b52-27ff-469a-b7d2-00971fbfd698 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:21.854676 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 938/1822] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:23.882427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b65604-7ab3-4fdf-bf27-a1ad41813667 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:23.886767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-97b65604-7ab3-4fdf-bf27-a1ad41813667 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:23.905600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b43b52-27ff-469a-b7d2-00971fbfd698 service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:23.907273 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-96b43b52-27ff-469a-b7d2-00971fbfd698 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:23.931672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b43b52-27ff-469a-b7d2-00971fbfd698 service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:23.932676 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b65604-7ab3-4fdf-bf27-a1ad41813667 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:23.937235 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5248a-866f-497c-b8f6-f8e9af4fbc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:23.937915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5248a-866f-497c-b8f6-f8e9af4fbc9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:23.938203 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5248a-866f-497c-b8f6-f8e9af4fbc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:23.938482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5248a-866f-497c-b8f6-f8e9af4fbc9b admin admin] PATH_INFO: `/v3/projects/8c20e3253ec6494886386d64e7883e61` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:23.939681 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f5248a-866f-497c-b8f6-f8e9af4fbc9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c20e3253ec6494886386d64e7883e61)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:23.941749 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d603cd-c59f-4348-87f6-d310c4d20173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:23.942902 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d603cd-c59f-4348-87f6-d310c4d20173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:23.943281 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f5248a-866f-497c-b8f6-f8e9af4fbc9b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:23.943569 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d603cd-c59f-4348-87f6-d310c4d20173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:23.944054 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d603cd-c59f-4348-87f6-d310c4d20173 admin admin] PATH_INFO: `/v3/projects/9edf8e6e05cc49f386e6a77afd7c9f08` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:23.945468 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d603cd-c59f-4348-87f6-d310c4d20173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9edf8e6e05cc49f386e6a77afd7c9f08)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:23.947507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d603cd-c59f-4348-87f6-d310c4d20173 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:24.012625 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-72f5248a-866f-497c-b8f6-f8e9af4fbc9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c20e3253ec6494886386d64e7883e61', 'request_id': 'req-72f5248a-866f-497c-b8f6-f8e9af4fbc9b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.020324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-98d603cd-c59f-4348-87f6-d310c4d20173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9edf8e6e05cc49f386e6a77afd7c9f08', 'request_id': 'req-98d603cd-c59f-4348-87f6-d310c4d20173'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.020933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-72f5248a-866f-497c-b8f6-f8e9af4fbc9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c20e3253ec6494886386d64e7883e61', 'request_id': 'req-72f5248a-866f-497c-b8f6-f8e9af4fbc9b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.027225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-98d603cd-c59f-4348-87f6-d310c4d20173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9edf8e6e05cc49f386e6a77afd7c9f08', 'request_id': 'req-98d603cd-c59f-4348-87f6-d310c4d20173'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.028244 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-72f5248a-866f-497c-b8f6-f8e9af4fbc9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c20e3253ec6494886386d64e7883e61', 'request_id': 'req-72f5248a-866f-497c-b8f6-f8e9af4fbc9b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.037262 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-98d603cd-c59f-4348-87f6-d310c4d20173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9edf8e6e05cc49f386e6a77afd7c9f08', 'request_id': 'req-98d603cd-c59f-4348-87f6-d310c4d20173'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.042545 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-72f5248a-866f-497c-b8f6-f8e9af4fbc9b admin admin] The token cache is being invalidate because project 8c20e3253ec6494886386d64e7883e61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:24.042918 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-72f5248a-866f-497c-b8f6-f8e9af4fbc9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.043599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-72f5248a-866f-497c-b8f6-f8e9af4fbc9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.045240 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 939/1823] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:23 2022] DELETE /identity/v3/projects/8c20e3253ec6494886386d64e7883e61 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:24.051130 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-98d603cd-c59f-4348-87f6-d310c4d20173 admin admin] The token cache is being invalidate because project 9edf8e6e05cc49f386e6a77afd7c9f08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:24.051428 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-98d603cd-c59f-4348-87f6-d310c4d20173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.052311 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-98d603cd-c59f-4348-87f6-d310c4d20173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.052790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5248a-866f-497c-b8f6-f8e9af4fbc9b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:24.053830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 885/1824] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:23 2022] DELETE /identity/v3/projects/9edf8e6e05cc49f386e6a77afd7c9f08 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:24.055792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-72f5248a-866f-497c-b8f6-f8e9af4fbc9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:24.057655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d603cd-c59f-4348-87f6-d310c4d20173 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:24.059079 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-98d603cd-c59f-4348-87f6-d310c4d20173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:24.129686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d603cd-c59f-4348-87f6-d310c4d20173 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:24.134350 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4af39-2256-4465-8888-6670a82d7afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:24.134984 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4af39-2256-4465-8888-6670a82d7afb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:24.135268 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4af39-2256-4465-8888-6670a82d7afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:24.135544 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4af39-2256-4465-8888-6670a82d7afb admin admin] PATH_INFO: `/v3/users/d608c3b8f285426e8d1aa34a98338d6f` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:24.136851 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4af39-2256-4465-8888-6670a82d7afb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d608c3b8f285426e8d1aa34a98338d6f)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:24.138832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4af39-2256-4465-8888-6670a82d7afb admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:24.141159 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5248a-866f-497c-b8f6-f8e9af4fbc9b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:24.144312 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:24.146230 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9418eaa-ad05-4ce0-92c5-afbaf3806404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:24.147007 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9418eaa-ad05-4ce0-92c5-afbaf3806404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:24.147293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9418eaa-ad05-4ce0-92c5-afbaf3806404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:24.147565 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9418eaa-ad05-4ce0-92c5-afbaf3806404 admin admin] PATH_INFO: `/v3/users/60a43d653a234f96bf4b9bdd250d15a8` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:24.148782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9418eaa-ad05-4ce0-92c5-afbaf3806404 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60a43d653a234f96bf4b9bdd250d15a8)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:24.151034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9418eaa-ad05-4ce0-92c5-afbaf3806404 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:24.192258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:24.194545 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c8f4af39-2256-4465-8888-6670a82d7afb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd608c3b8f285426e8d1aa34a98338d6f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.202905 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c8f4af39-2256-4465-8888-6670a82d7afb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd608c3b8f285426e8d1aa34a98338d6f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.204180 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b9418eaa-ad05-4ce0-92c5-afbaf3806404 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60a43d653a234f96bf4b9bdd250d15a8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.211114 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c8f4af39-2256-4465-8888-6670a82d7afb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd608c3b8f285426e8d1aa34a98338d6f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.211835 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c8f4af39-2256-4465-8888-6670a82d7afb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd608c3b8f285426e8d1aa34a98338d6f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.212251 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b9418eaa-ad05-4ce0-92c5-afbaf3806404 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60a43d653a234f96bf4b9bdd250d15a8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.221009 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b9418eaa-ad05-4ce0-92c5-afbaf3806404 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60a43d653a234f96bf4b9bdd250d15a8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.222199 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b9418eaa-ad05-4ce0-92c5-afbaf3806404 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60a43d653a234f96bf4b9bdd250d15a8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.222852 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b9418eaa-ad05-4ce0-92c5-afbaf3806404 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60a43d653a234f96bf4b9bdd250d15a8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.226657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c8f4af39-2256-4465-8888-6670a82d7afb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd608c3b8f285426e8d1aa34a98338d6f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.232704 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 886/1825] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:24 2022] DELETE /identity/v3/users/d608c3b8f285426e8d1aa34a98338d6f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:24.241951 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 940/1826] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:24 2022] DELETE /identity/v3/users/60a43d653a234f96bf4b9bdd250d15a8 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:24.540323 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4af39-2256-4465-8888-6670a82d7afb admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:24.542219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c8f4af39-2256-4465-8888-6670a82d7afb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:24.622435 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4af39-2256-4465-8888-6670a82d7afb admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:24.677675 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9418eaa-ad05-4ce0-92c5-afbaf3806404 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:24.679218 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b9418eaa-ad05-4ce0-92c5-afbaf3806404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:24.685141 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25baef2b-8d00-46e2-acb9-ea729eaa22b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:24.685789 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25baef2b-8d00-46e2-acb9-ea729eaa22b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:24.686073 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25baef2b-8d00-46e2-acb9-ea729eaa22b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:24.686349 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25baef2b-8d00-46e2-acb9-ea729eaa22b7 admin admin] PATH_INFO: `/v3/projects/c1a336ba0d2c46cb91eb774adb7ef471` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:24.687480 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25baef2b-8d00-46e2-acb9-ea729eaa22b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1a336ba0d2c46cb91eb774adb7ef471)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:24.689452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25baef2b-8d00-46e2-acb9-ea729eaa22b7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:24.696762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9418eaa-ad05-4ce0-92c5-afbaf3806404 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:24.703723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be3d50-27cb-48fc-9e7f-a7fb9f8e71f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:24.704575 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be3d50-27cb-48fc-9e7f-a7fb9f8e71f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:24.704862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be3d50-27cb-48fc-9e7f-a7fb9f8e71f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:24.705207 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be3d50-27cb-48fc-9e7f-a7fb9f8e71f8 admin admin] PATH_INFO: `/v3/projects/2fd6925eae174e54ae1ac0464ba14f51` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:24.706937 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be3d50-27cb-48fc-9e7f-a7fb9f8e71f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fd6925eae174e54ae1ac0464ba14f51)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:24.709068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be3d50-27cb-48fc-9e7f-a7fb9f8e71f8 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:24.734395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:24.747774 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-25baef2b-8d00-46e2-acb9-ea729eaa22b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1a336ba0d2c46cb91eb774adb7ef471', 'request_id': 'req-25baef2b-8d00-46e2-acb9-ea729eaa22b7'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.756010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-25baef2b-8d00-46e2-acb9-ea729eaa22b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1a336ba0d2c46cb91eb774adb7ef471', 'request_id': 'req-25baef2b-8d00-46e2-acb9-ea729eaa22b7'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.764239 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-25baef2b-8d00-46e2-acb9-ea729eaa22b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1a336ba0d2c46cb91eb774adb7ef471', 'request_id': 'req-25baef2b-8d00-46e2-acb9-ea729eaa22b7'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.771351 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c8be3d50-27cb-48fc-9e7f-a7fb9f8e71f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fd6925eae174e54ae1ac0464ba14f51', 'request_id': 'req-c8be3d50-27cb-48fc-9e7f-a7fb9f8e71f8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.779732 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c8be3d50-27cb-48fc-9e7f-a7fb9f8e71f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fd6925eae174e54ae1ac0464ba14f51', 'request_id': 'req-c8be3d50-27cb-48fc-9e7f-a7fb9f8e71f8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.780774 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-25baef2b-8d00-46e2-acb9-ea729eaa22b7 admin admin] The token cache is being invalidate because project c1a336ba0d2c46cb91eb774adb7ef471 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:24.781259 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-25baef2b-8d00-46e2-acb9-ea729eaa22b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.782053 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-25baef2b-8d00-46e2-acb9-ea729eaa22b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.788042 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 887/1827] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:24 2022] DELETE /identity/v3/projects/c1a336ba0d2c46cb91eb774adb7ef471 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:24.789472 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c8be3d50-27cb-48fc-9e7f-a7fb9f8e71f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fd6925eae174e54ae1ac0464ba14f51', 'request_id': 'req-c8be3d50-27cb-48fc-9e7f-a7fb9f8e71f8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.790523 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b9eca-24d5-49f9-b042-ea01bcc0076d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:24.791187 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b9eca-24d5-49f9-b042-ea01bcc0076d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:24.791628 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b9eca-24d5-49f9-b042-ea01bcc0076d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:24.807265 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c8be3d50-27cb-48fc-9e7f-a7fb9f8e71f8 admin admin] The token cache is being invalidate because project 2fd6925eae174e54ae1ac0464ba14f51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:24.807689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c8be3d50-27cb-48fc-9e7f-a7fb9f8e71f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.809371 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c8be3d50-27cb-48fc-9e7f-a7fb9f8e71f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:24.816370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e36b9eca-24d5-49f9-b042-ea01bcc0076d None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:24.818034 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e36b9eca-24d5-49f9-b042-ea01bcc0076d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:24.828027 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 941/1828] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:24 2022] DELETE /identity/v3/projects/2fd6925eae174e54ae1ac0464ba14f51 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:24.834874 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e6e16-0482-4896-bf2d-ce746171dae4 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:24.835541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e6e16-0482-4896-bf2d-ce746171dae4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:24.836267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e6e16-0482-4896-bf2d-ce746171dae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:24.869938 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a65e6e16-0482-4896-bf2d-ce746171dae4 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:24.870675 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a65e6e16-0482-4896-bf2d-ce746171dae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:24.923616 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 888/1829] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:24.932555 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2daf99-c12c-4831-ab1c-e20dc15d81e0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:24.933365 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2daf99-c12c-4831-ab1c-e20dc15d81e0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:24.933890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2daf99-c12c-4831-ab1c-e20dc15d81e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:24.959274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 942/1830] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:24.968531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d6110-b3ed-461b-9083-5351aa20f6d5 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:24.968827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d6110-b3ed-461b-9083-5351aa20f6d5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:24.969187 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d6110-b3ed-461b-9083-5351aa20f6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:24.970291 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-4d2daf99-c12c-4831-ab1c-e20dc15d81e0 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:24.971774 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4d2daf99-c12c-4831-ab1c-e20dc15d81e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:24.996602 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 889/1831] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:25.000942 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2daf99-c12c-4831-ab1c-e20dc15d81e0 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:25.005086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-549d6110-b3ed-461b-9083-5351aa20f6d5 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:25.010995 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-549d6110-b3ed-461b-9083-5351aa20f6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:25.012503 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2daf99-c12c-4831-ab1c-e20dc15d81e0 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:25.020726 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53513d4-31ca-4a47-892e-8402ec6c84f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:25.021349 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53513d4-31ca-4a47-892e-8402ec6c84f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:25.021629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53513d4-31ca-4a47-892e-8402ec6c84f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:25.021923 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53513d4-31ca-4a47-892e-8402ec6c84f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:25.022985 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53513d4-31ca-4a47-892e-8402ec6c84f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:33:25.023284 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53513d4-31ca-4a47-892e-8402ec6c84f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:25.024903 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53513d4-31ca-4a47-892e-8402ec6c84f5 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:25.027479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 943/1832] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:25.032445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d6110-b3ed-461b-9083-5351aa20f6d5 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:25.047859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d6110-b3ed-461b-9083-5351aa20f6d5 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:25.066288 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 890/1833] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:33:25.074558 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8178851-2665-4bef-aa09-0786b347a964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:25.075279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8178851-2665-4bef-aa09-0786b347a964 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:25.075576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8178851-2665-4bef-aa09-0786b347a964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:25.076191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8178851-2665-4bef-aa09-0786b347a964 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:25.076590 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8178851-2665-4bef-aa09-0786b347a964 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:33:25.077049 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8178851-2665-4bef-aa09-0786b347a964 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:25.078739 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8178851-2665-4bef-aa09-0786b347a964 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:25.082848 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53513d4-31ca-4a47-892e-8402ec6c84f5 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:25.103330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 944/1834] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:33:25.104307 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8178851-2665-4bef-aa09-0786b347a964 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:25.119277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8178851-2665-4bef-aa09-0786b347a964 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:25.121594 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53513d4-31ca-4a47-892e-8402ec6c84f5 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:25.138115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed9d44-7583-41e5-9f8c-1a1c63e187f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:25.138740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed9d44-7583-41e5-9f8c-1a1c63e187f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:25.139019 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed9d44-7583-41e5-9f8c-1a1c63e187f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:25.139289 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed9d44-7583-41e5-9f8c-1a1c63e187f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:25.139922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed9d44-7583-41e5-9f8c-1a1c63e187f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:25.141862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed9d44-7583-41e5-9f8c-1a1c63e187f2 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:25.146606 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfacd6-d4e7-48ea-af4c-2c677b3f4c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:25.148182 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfacd6-d4e7-48ea-af4c-2c677b3f4c77 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:25.148775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfacd6-d4e7-48ea-af4c-2c677b3f4c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:25.149799 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfacd6-d4e7-48ea-af4c-2c677b3f4c77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:25.151062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfacd6-d4e7-48ea-af4c-2c677b3f4c77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:25.158193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dfacd6-d4e7-48ea-af4c-2c677b3f4c77 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:25.195178 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:25.220006 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 891/1835] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:25 2022] POST /identity/v3/projects => generated 357 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:25.225744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed9d44-7583-41e5-9f8c-1a1c63e187f2 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:25.256097 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 945/1836] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:25 2022] POST /identity/v3/projects => generated 387 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:25.267203 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfacd6-d4e7-48ea-af4c-2c677b3f4c77 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:25.357586 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed9d44-7583-41e5-9f8c-1a1c63e187f2 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:25.432919 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:25.437245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfacd6-d4e7-48ea-af4c-2c677b3f4c77 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:25.445949 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953ba18-550a-4893-8b1c-ff386042516c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:25.446708 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953ba18-550a-4893-8b1c-ff386042516c admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:25.446993 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953ba18-550a-4893-8b1c-ff386042516c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:25.447426 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953ba18-550a-4893-8b1c-ff386042516c admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:25.448018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953ba18-550a-4893-8b1c-ff386042516c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:25.449946 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953ba18-550a-4893-8b1c-ff386042516c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:25.480365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 946/1837] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:25 2022] POST /identity/v3/users => generated 294 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:25.492051 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953ba18-550a-4893-8b1c-ff386042516c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:25.508030 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953ba18-550a-4893-8b1c-ff386042516c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:25.508750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23501143-8c67-4d2f-af7d-b04a28b800fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:25.509481 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23501143-8c67-4d2f-af7d-b04a28b800fe admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:25.509773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23501143-8c67-4d2f-af7d-b04a28b800fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:25.510129 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23501143-8c67-4d2f-af7d-b04a28b800fe admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:25.510716 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23501143-8c67-4d2f-af7d-b04a28b800fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:25.512767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23501143-8c67-4d2f-af7d-b04a28b800fe admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:25.514217 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e912355-84d6-403b-a0b0-67e58b15d4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:25.514934 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e912355-84d6-403b-a0b0-67e58b15d4c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:25.515279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e912355-84d6-403b-a0b0-67e58b15d4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:25.515770 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e912355-84d6-403b-a0b0-67e58b15d4c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:25.516548 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e912355-84d6-403b-a0b0-67e58b15d4c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:25.518433 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e912355-84d6-403b-a0b0-67e58b15d4c4 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:25.533577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 947/1838] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:25.537395 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e912355-84d6-403b-a0b0-67e58b15d4c4 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:25.541415 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 892/1839] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:25 2022] POST /identity/v3/users => generated 279 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:25.545383 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23501143-8c67-4d2f-af7d-b04a28b800fe admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:25.548208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e912355-84d6-403b-a0b0-67e58b15d4c4 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:25.556211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1a02a-5460-428a-9f9e-00042cb4ad73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:25.558443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1a02a-5460-428a-9f9e-00042cb4ad73 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:25.558944 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1a02a-5460-428a-9f9e-00042cb4ad73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:25.559581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1a02a-5460-428a-9f9e-00042cb4ad73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:25.560404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23501143-8c67-4d2f-af7d-b04a28b800fe admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:25.560992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1a02a-5460-428a-9f9e-00042cb4ad73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:25.563223 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1a02a-5460-428a-9f9e-00042cb4ad73 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:25.575062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97715936-766b-4227-859f-58ffd26b2747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:25.575687 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97715936-766b-4227-859f-58ffd26b2747 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:25.575998 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97715936-766b-4227-859f-58ffd26b2747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:25.576275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97715936-766b-4227-859f-58ffd26b2747 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:25.576991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97715936-766b-4227-859f-58ffd26b2747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:25.578820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97715936-766b-4227-859f-58ffd26b2747 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:25.582619 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 948/1840] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:33:25.586208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1a02a-5460-428a-9f9e-00042cb4ad73 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:25.595988 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 893/1841] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:25.602776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97715936-766b-4227-859f-58ffd26b2747 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:25.603362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1a02a-5460-428a-9f9e-00042cb4ad73 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:25.609495 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c9674-7bba-4ada-99c2-e059363bbb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:25.613753 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97715936-766b-4227-859f-58ffd26b2747 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:25.613933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c9674-7bba-4ada-99c2-e059363bbb2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:25.614208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c9674-7bba-4ada-99c2-e059363bbb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:25.614503 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c9674-7bba-4ada-99c2-e059363bbb2c admin admin] PATH_INFO: `/v3/projects/57500ff8f5a948efa78d908d3a7fb417/users/200a49c9ee19485f838e23bfa8f21b8f/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:25.628370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90288ff1-0744-4566-bded-a3fe098d9c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:25.629252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90288ff1-0744-4566-bded-a3fe098d9c64 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:25.630235 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90288ff1-0744-4566-bded-a3fe098d9c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:25.630810 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90288ff1-0744-4566-bded-a3fe098d9c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:25.632138 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90288ff1-0744-4566-bded-a3fe098d9c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:25.634501 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90288ff1-0744-4566-bded-a3fe098d9c64 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:25.647612 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c9674-7bba-4ada-99c2-e059363bbb2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=57500ff8f5a948efa78d908d3a7fb417, user_id=200a49c9ee19485f838e23bfa8f21b8f, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:25.655777 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 894/1842] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:25.656436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c9674-7bba-4ada-99c2-e059363bbb2c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:25.657680 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90288ff1-0744-4566-bded-a3fe098d9c64 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:25.672369 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90288ff1-0744-4566-bded-a3fe098d9c64 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:25.676125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 949/1843] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:25 2022] PUT /identity/v3/projects/57500ff8f5a948efa78d908d3a7fb417/users/200a49c9ee19485f838e23bfa8f21b8f/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:25.685041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6533ad-72f2-4843-8d1d-edc5cb59efee None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:25.685885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6533ad-72f2-4843-8d1d-edc5cb59efee None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:25.686475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6533ad-72f2-4843-8d1d-edc5cb59efee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:25.727008 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ba6533ad-72f2-4843-8d1d-edc5cb59efee None None] MFA Rules not processed for user `200a49c9ee19485f838e23bfa8f21b8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:25.729126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ba6533ad-72f2-4843-8d1d-edc5cb59efee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:25.758856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04cdbd-bad3-4f80-b995-babe9f1a9280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:25.759721 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04cdbd-bad3-4f80-b995-babe9f1a9280 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:25.760076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04cdbd-bad3-4f80-b995-babe9f1a9280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:25.760387 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04cdbd-bad3-4f80-b995-babe9f1a9280 admin admin] PATH_INFO: `/v3/projects/1268ac632b7d43289d77234d055d537c/users/17f987e2b97849bca2e92bbce21f2674/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:25.779329 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04cdbd-bad3-4f80-b995-babe9f1a9280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1268ac632b7d43289d77234d055d537c, user_id=17f987e2b97849bca2e92bbce21f2674, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:25.782131 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04cdbd-bad3-4f80-b995-babe9f1a9280 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:25.796680 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 895/1844] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:25 2022] PUT /identity/v3/projects/1268ac632b7d43289d77234d055d537c/users/17f987e2b97849bca2e92bbce21f2674/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:25.813806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b701fd-a38d-4966-8d5f-f1e4af56af65 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:25.814103 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b701fd-a38d-4966-8d5f-f1e4af56af65 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:25.814382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b701fd-a38d-4966-8d5f-f1e4af56af65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:25.829231 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 950/1845] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:25.835298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6533ad-72f2-4843-8d1d-edc5cb59efee None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:25.848942 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-74b701fd-a38d-4966-8d5f-f1e4af56af65 None None] MFA Rules not processed for user `17f987e2b97849bca2e92bbce21f2674`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:25.850187 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-74b701fd-a38d-4966-8d5f-f1e4af56af65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:25.851680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6533ad-72f2-4843-8d1d-edc5cb59efee None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:25.886775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:25.926283 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473de6aa-b29c-4575-ad60-281788ae0088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:25.927340 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473de6aa-b29c-4575-ad60-281788ae0088 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:25.927681 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473de6aa-b29c-4575-ad60-281788ae0088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:25.928153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473de6aa-b29c-4575-ad60-281788ae0088 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:25.929228 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473de6aa-b29c-4575-ad60-281788ae0088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:25.932344 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473de6aa-b29c-4575-ad60-281788ae0088 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:25.942517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 896/1846] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:25 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:25.949389 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b701fd-a38d-4966-8d5f-f1e4af56af65 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:25.971372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b701fd-a38d-4966-8d5f-f1e4af56af65 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:25.976325 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 951/1847] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:25 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:25.979284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473de6aa-b29c-4575-ad60-281788ae0088 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:25.981354 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:26.100098 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e2395-677c-4b5e-9f71-7b802f651095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:26.100731 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e2395-677c-4b5e-9f71-7b802f651095 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:26.101022 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e2395-677c-4b5e-9f71-7b802f651095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:26.101298 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e2395-677c-4b5e-9f71-7b802f651095 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:26.101972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e2395-677c-4b5e-9f71-7b802f651095 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:26.103898 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e2395-677c-4b5e-9f71-7b802f651095 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:26.104892 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473de6aa-b29c-4575-ad60-281788ae0088 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:26.111373 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715fc54e-d96c-410b-a3a0-a0c28b83a288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:26.113669 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fc54e-d96c-410b-a3a0-a0c28b83a288 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:26.114100 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fc54e-d96c-410b-a3a0-a0c28b83a288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:26.114529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fc54e-d96c-410b-a3a0-a0c28b83a288 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:26.115313 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715fc54e-d96c-410b-a3a0-a0c28b83a288 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:26.118702 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715fc54e-d96c-410b-a3a0-a0c28b83a288 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:26.149106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 952/1848] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:25 2022] POST /identity/v3/users => generated 299 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:26.152942 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715fc54e-d96c-410b-a3a0-a0c28b83a288 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:26.154856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 897/1849] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:25 2022] POST /identity/v3/projects => generated 357 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:26.158921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e2395-677c-4b5e-9f71-7b802f651095 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:26.180411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715fc54e-d96c-410b-a3a0-a0c28b83a288 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:26.256153 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e2395-677c-4b5e-9f71-7b802f651095 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:26.271142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240881e-dd81-417e-b82a-c9c27ec586ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:26.271772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240881e-dd81-417e-b82a-c9c27ec586ee admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:26.272081 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240881e-dd81-417e-b82a-c9c27ec586ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:26.272736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240881e-dd81-417e-b82a-c9c27ec586ee admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:26.273001 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b240881e-dd81-417e-b82a-c9c27ec586ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:26.274939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b240881e-dd81-417e-b82a-c9c27ec586ee admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:26.283868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd4dc8-1dcb-4b6b-af5c-6002ffaf62bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:26.285087 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd4dc8-1dcb-4b6b-af5c-6002ffaf62bf admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:26.285376 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd4dc8-1dcb-4b6b-af5c-6002ffaf62bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:26.285656 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fd4dc8-1dcb-4b6b-af5c-6002ffaf62bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:26.286493 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd4dc8-1dcb-4b6b-af5c-6002ffaf62bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:26.288750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fd4dc8-1dcb-4b6b-af5c-6002ffaf62bf admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:26.305794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 898/1850] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:26 2022] POST /identity/v3/users => generated 285 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:26.306755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240881e-dd81-417e-b82a-c9c27ec586ee admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:26.318512 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 953/1851] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:26.322659 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd4dc8-1dcb-4b6b-af5c-6002ffaf62bf admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:26.326588 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240881e-dd81-417e-b82a-c9c27ec586ee admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:26.349092 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fd4dc8-1dcb-4b6b-af5c-6002ffaf62bf admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:26.361907 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8304de0-37dc-402b-863a-5e9c26f2038f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:26.361907 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8304de0-37dc-402b-863a-5e9c26f2038f admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:26.361907 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8304de0-37dc-402b-863a-5e9c26f2038f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:26.361907 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8304de0-37dc-402b-863a-5e9c26f2038f admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:26.362450 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8304de0-37dc-402b-863a-5e9c26f2038f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:26.362450 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8304de0-37dc-402b-863a-5e9c26f2038f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:26.362584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e10a01-7657-4e6e-9bf1-aeaa6b0ab2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:26.362584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e10a01-7657-4e6e-9bf1-aeaa6b0ab2c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:26.362584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e10a01-7657-4e6e-9bf1-aeaa6b0ab2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:26.362584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e10a01-7657-4e6e-9bf1-aeaa6b0ab2c9 admin admin] PATH_INFO: `/v3/projects/84cfffbe1f374cafa4f9925cc14cd256/users/11a77abcb00e4050a19957e2de2734a4/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:26.380292 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e10a01-7657-4e6e-9bf1-aeaa6b0ab2c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84cfffbe1f374cafa4f9925cc14cd256, user_id=11a77abcb00e4050a19957e2de2734a4, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:26.383144 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e10a01-7657-4e6e-9bf1-aeaa6b0ab2c9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:26.386853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 899/1852] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:26.389088 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8304de0-37dc-402b-863a-5e9c26f2038f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:26.403481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 954/1853] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:26 2022] PUT /identity/v3/projects/84cfffbe1f374cafa4f9925cc14cd256/users/11a77abcb00e4050a19957e2de2734a4/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:26.404868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e10a01-7657-4e6e-9bf1-aeaa6b0ab2c9 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:26.413531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8304de0-37dc-402b-863a-5e9c26f2038f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:26.421253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:26.492118 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290ce51-ef0a-4e52-bf59-2db7d422b4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:26.492815 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290ce51-ef0a-4e52-bf59-2db7d422b4b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:26.493087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290ce51-ef0a-4e52-bf59-2db7d422b4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:26.493361 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290ce51-ef0a-4e52-bf59-2db7d422b4b0 admin admin] PATH_INFO: `/v3/projects/1612b88af26d41158cee3dbcd8f54e24/users/aa0aefbcfb2740cfad0c0cd6b7e1a031/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:26.511711 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e10a01-7657-4e6e-9bf1-aeaa6b0ab2c9 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:26.515103 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290ce51-ef0a-4e52-bf59-2db7d422b4b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1612b88af26d41158cee3dbcd8f54e24, user_id=aa0aefbcfb2740cfad0c0cd6b7e1a031, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:26.516945 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cd022-da98-4753-958b-c4b98143db5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:26.531159 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290ce51-ef0a-4e52-bf59-2db7d422b4b0 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:26.531159 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:26.531481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cd022-da98-4753-958b-c4b98143db5d admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:26.531481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cd022-da98-4753-958b-c4b98143db5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:26.531481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cd022-da98-4753-958b-c4b98143db5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:26.531481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8cd022-da98-4753-958b-c4b98143db5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:26.531481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8cd022-da98-4753-958b-c4b98143db5d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:26.543547 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 900/1854] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:26 2022] PUT /identity/v3/projects/1612b88af26d41158cee3dbcd8f54e24/users/aa0aefbcfb2740cfad0c0cd6b7e1a031/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:26.544814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290ce51-ef0a-4e52-bf59-2db7d422b4b0 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:26.551756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 955/1855] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:26.553216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cd022-da98-4753-958b-c4b98143db5d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:26.572057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cd022-da98-4753-958b-c4b98143db5d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:26.636571 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290ce51-ef0a-4e52-bf59-2db7d422b4b0 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:26.643806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb348e2c-eeb8-44f7-b261-0e20d8c41604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:26.655381 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23be84e-41eb-41c9-bcc0-e0165c742010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:26.655915 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb348e2c-eeb8-44f7-b261-0e20d8c41604 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:26.656369 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb348e2c-eeb8-44f7-b261-0e20d8c41604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:26.656647 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23be84e-41eb-41c9-bcc0-e0165c742010 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:26.657200 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23be84e-41eb-41c9-bcc0-e0165c742010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:26.657776 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23be84e-41eb-41c9-bcc0-e0165c742010 admin admin] PATH_INFO: `/v3/projects/84cfffbe1f374cafa4f9925cc14cd256/users/11a77abcb00e4050a19957e2de2734a4/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:26.657986 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb348e2c-eeb8-44f7-b261-0e20d8c41604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:26.658704 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb348e2c-eeb8-44f7-b261-0e20d8c41604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:26.660719 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb348e2c-eeb8-44f7-b261-0e20d8c41604 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:26.661585 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23be84e-41eb-41c9-bcc0-e0165c742010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84cfffbe1f374cafa4f9925cc14cd256, user_id=11a77abcb00e4050a19957e2de2734a4, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:26.665650 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23be84e-41eb-41c9-bcc0-e0165c742010 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:26.672947 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 901/1856] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:26.677276 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb348e2c-eeb8-44f7-b261-0e20d8c41604 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:26.678986 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 956/1857] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:26 2022] PUT /identity/v3/projects/84cfffbe1f374cafa4f9925cc14cd256/users/11a77abcb00e4050a19957e2de2734a4/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:26.692049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb348e2c-eeb8-44f7-b261-0e20d8c41604 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:26.695180 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd636a-21dc-4723-8772-87f14d9525dd None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:26.695694 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd636a-21dc-4723-8772-87f14d9525dd None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:26.696125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd636a-21dc-4723-8772-87f14d9525dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:26.741687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-9efd636a-21dc-4723-8772-87f14d9525dd None None] MFA Rules not processed for user `11a77abcb00e4050a19957e2de2734a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:26.744443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9efd636a-21dc-4723-8772-87f14d9525dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:26.765258 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e58af-0df8-4c8e-8464-9d75cc7b9450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:26.766070 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e58af-0df8-4c8e-8464-9d75cc7b9450 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:26.766590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e58af-0df8-4c8e-8464-9d75cc7b9450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:26.766923 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e58af-0df8-4c8e-8464-9d75cc7b9450 admin admin] PATH_INFO: `/v3/projects/1612b88af26d41158cee3dbcd8f54e24/users/aa0aefbcfb2740cfad0c0cd6b7e1a031/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:26.770152 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e58af-0df8-4c8e-8464-9d75cc7b9450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1612b88af26d41158cee3dbcd8f54e24, user_id=aa0aefbcfb2740cfad0c0cd6b7e1a031, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:26.773455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e58af-0df8-4c8e-8464-9d75cc7b9450 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:26.796096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 902/1858] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:26 2022] PUT /identity/v3/projects/1612b88af26d41158cee3dbcd8f54e24/users/aa0aefbcfb2740cfad0c0cd6b7e1a031/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:26.819853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b91f33-5486-49d7-aa4f-f25555be69c5 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:26.822002 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b91f33-5486-49d7-aa4f-f25555be69c5 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:26.822795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b91f33-5486-49d7-aa4f-f25555be69c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:26.871802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-81b91f33-5486-49d7-aa4f-f25555be69c5 None None] MFA Rules not processed for user `aa0aefbcfb2740cfad0c0cd6b7e1a031`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:26.873289 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-81b91f33-5486-49d7-aa4f-f25555be69c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:26.876192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 957/1859] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:26.899922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd636a-21dc-4723-8772-87f14d9525dd None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:26.901641 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9efd636a-21dc-4723-8772-87f14d9525dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:26.967979 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:27.017152 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 903/1860] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:26 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:27.023623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd636a-21dc-4723-8772-87f14d9525dd None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:27.056391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b91f33-5486-49d7-aa4f-f25555be69c5 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:27.058205 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-81b91f33-5486-49d7-aa4f-f25555be69c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:27.075833 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:27.101101 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc6601-1b79-4d1d-8a33-97fbbf5d6cbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:27.101730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc6601-1b79-4d1d-8a33-97fbbf5d6cbc service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:27.102023 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc6601-1b79-4d1d-8a33-97fbbf5d6cbc service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:27.102297 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc6601-1b79-4d1d-8a33-97fbbf5d6cbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:27.144751 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b91f33-5486-49d7-aa4f-f25555be69c5 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:27.151581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df16af-dd0d-4134-afda-31fddd2e3891 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:27.152940 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df16af-dd0d-4134-afda-31fddd2e3891 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:27.153259 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df16af-dd0d-4134-afda-31fddd2e3891 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:27.153577 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df16af-dd0d-4134-afda-31fddd2e3891 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:27.166435 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df16af-dd0d-4134-afda-31fddd2e3891 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:27.168175 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df16af-dd0d-4134-afda-31fddd2e3891 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:27.183038 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 904/1861] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:27 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:27.216944 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc6601-1b79-4d1d-8a33-97fbbf5d6cbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:27.219513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc6601-1b79-4d1d-8a33-97fbbf5d6cbc service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:27.233202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 958/1862] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:27.265235 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df16af-dd0d-4134-afda-31fddd2e3891 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:27.345875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc6601-1b79-4d1d-8a33-97fbbf5d6cbc service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:27.365410 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc6601-1b79-4d1d-8a33-97fbbf5d6cbc service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:27.386408 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573ead8-cbf0-4545-86fa-cf95d562e478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:27.387734 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573ead8-cbf0-4545-86fa-cf95d562e478 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:27.389122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573ead8-cbf0-4545-86fa-cf95d562e478 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:27.390708 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573ead8-cbf0-4545-86fa-cf95d562e478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:27.510196 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573ead8-cbf0-4545-86fa-cf95d562e478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:27.521979 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573ead8-cbf0-4545-86fa-cf95d562e478 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:27.538115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df16af-dd0d-4134-afda-31fddd2e3891 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:27.555485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 959/1863] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:27 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:27.555702 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccfc48-aa99-4fea-8789-050862934dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:27.556409 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccfc48-aa99-4fea-8789-050862934dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:27.557094 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccfc48-aa99-4fea-8789-050862934dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:27.557094 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccfc48-aa99-4fea-8789-050862934dce admin admin] PATH_INFO: `/v3/users/200a49c9ee19485f838e23bfa8f21b8f` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:27.572818 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccfc48-aa99-4fea-8789-050862934dce admin admin] RBAC: Authorizing `identity:delete_user(user_id=200a49c9ee19485f838e23bfa8f21b8f)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:27.580884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccfc48-aa99-4fea-8789-050862934dce admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:27.748037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-01ccfc48-aa99-4fea-8789-050862934dce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '200a49c9ee19485f838e23bfa8f21b8f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:27.758204 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-01ccfc48-aa99-4fea-8789-050862934dce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '200a49c9ee19485f838e23bfa8f21b8f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:27.782708 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-01ccfc48-aa99-4fea-8789-050862934dce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '200a49c9ee19485f838e23bfa8f21b8f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:27.783754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-01ccfc48-aa99-4fea-8789-050862934dce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '200a49c9ee19485f838e23bfa8f21b8f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:27.840229 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-01ccfc48-aa99-4fea-8789-050862934dce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '200a49c9ee19485f838e23bfa8f21b8f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:27.872862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 905/1864] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:27 2022] DELETE /identity/v3/users/200a49c9ee19485f838e23bfa8f21b8f => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:27.874065 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b32d3-641e-47ef-b22c-caa2a47b04f0 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:27.874434 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b32d3-641e-47ef-b22c-caa2a47b04f0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:27.874844 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b32d3-641e-47ef-b22c-caa2a47b04f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:27.933791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-833b32d3-641e-47ef-b22c-caa2a47b04f0 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:27.938485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-833b32d3-641e-47ef-b22c-caa2a47b04f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:28.054112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:28.134558 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 960/1865] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:28.148200 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccfc48-aa99-4fea-8789-050862934dce admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:28.155444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-01ccfc48-aa99-4fea-8789-050862934dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:28.230989 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:28.320614 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccfc48-aa99-4fea-8789-050862934dce admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:28.337298 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e788877-9aab-4d67-b7e5-657b1ed714f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:28.338261 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e788877-9aab-4d67-b7e5-657b1ed714f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:28.338692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e788877-9aab-4d67-b7e5-657b1ed714f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:28.339125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e788877-9aab-4d67-b7e5-657b1ed714f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:28.371711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e788877-9aab-4d67-b7e5-657b1ed714f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:28.376846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e788877-9aab-4d67-b7e5-657b1ed714f4 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:28.399222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 906/1866] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:28.434328 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b32d3-641e-47ef-b22c-caa2a47b04f0 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:28.436318 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-833b32d3-641e-47ef-b22c-caa2a47b04f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:28.454222 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b32d3-641e-47ef-b22c-caa2a47b04f0 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:28.467760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff102d-03eb-42a0-b603-b15450a385d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:28.469135 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff102d-03eb-42a0-b603-b15450a385d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:28.469824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff102d-03eb-42a0-b603-b15450a385d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:28.471013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff102d-03eb-42a0-b603-b15450a385d3 admin admin] PATH_INFO: `/v3/users/0426d613bd51455792b6e52fe22bfd7b` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:28.473033 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff102d-03eb-42a0-b603-b15450a385d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0426d613bd51455792b6e52fe22bfd7b)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:28.489667 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff102d-03eb-42a0-b603-b15450a385d3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:28.619766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-acff102d-03eb-42a0-b603-b15450a385d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0426d613bd51455792b6e52fe22bfd7b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:28.629437 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-acff102d-03eb-42a0-b603-b15450a385d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0426d613bd51455792b6e52fe22bfd7b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:28.640090 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-acff102d-03eb-42a0-b603-b15450a385d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0426d613bd51455792b6e52fe22bfd7b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:28.641185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-acff102d-03eb-42a0-b603-b15450a385d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0426d613bd51455792b6e52fe22bfd7b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:28.641957 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-acff102d-03eb-42a0-b603-b15450a385d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0426d613bd51455792b6e52fe22bfd7b'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:28.693961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 961/1867] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:28 2022] DELETE /identity/v3/users/0426d613bd51455792b6e52fe22bfd7b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:28.699046 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c39ee4-a8d1-466f-ba65-4c51307babf3 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:28.699426 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c39ee4-a8d1-466f-ba65-4c51307babf3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:28.699658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c39ee4-a8d1-466f-ba65-4c51307babf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:28.758395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-79c39ee4-a8d1-466f-ba65-4c51307babf3 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:28.765021 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-79c39ee4-a8d1-466f-ba65-4c51307babf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:28.940488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 907/1868] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:28.952241 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff102d-03eb-42a0-b603-b15450a385d3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:28.953273 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-acff102d-03eb-42a0-b603-b15450a385d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:28.994698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c39ee4-a8d1-466f-ba65-4c51307babf3 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:28.996452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-79c39ee4-a8d1-466f-ba65-4c51307babf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:29.020057 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c39ee4-a8d1-466f-ba65-4c51307babf3 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:29.035585 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695c407-e617-4a1b-a9e7-c643cae73441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:29.036730 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695c407-e617-4a1b-a9e7-c643cae73441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:29.037335 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695c407-e617-4a1b-a9e7-c643cae73441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:29.037715 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695c407-e617-4a1b-a9e7-c643cae73441 admin admin] PATH_INFO: `/v3/projects/57500ff8f5a948efa78d908d3a7fb417` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:29.052846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695c407-e617-4a1b-a9e7-c643cae73441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57500ff8f5a948efa78d908d3a7fb417)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:29.061135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695c407-e617-4a1b-a9e7-c643cae73441 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:29.200782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff102d-03eb-42a0-b603-b15450a385d3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:29.242747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9695c407-e617-4a1b-a9e7-c643cae73441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57500ff8f5a948efa78d908d3a7fb417', 'request_id': 'req-9695c407-e617-4a1b-a9e7-c643cae73441'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:29.265644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9695c407-e617-4a1b-a9e7-c643cae73441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57500ff8f5a948efa78d908d3a7fb417', 'request_id': 'req-9695c407-e617-4a1b-a9e7-c643cae73441'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:29.280257 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9695c407-e617-4a1b-a9e7-c643cae73441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57500ff8f5a948efa78d908d3a7fb417', 'request_id': 'req-9695c407-e617-4a1b-a9e7-c643cae73441'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:29.333956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9695c407-e617-4a1b-a9e7-c643cae73441 admin admin] The token cache is being invalidate because project 57500ff8f5a948efa78d908d3a7fb417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:29.334196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9695c407-e617-4a1b-a9e7-c643cae73441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:29.336357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9695c407-e617-4a1b-a9e7-c643cae73441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:29.341699 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 908/1869] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:28 2022] DELETE /identity/v3/projects/57500ff8f5a948efa78d908d3a7fb417 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:29.347806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695c407-e617-4a1b-a9e7-c643cae73441 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:29.350890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9695c407-e617-4a1b-a9e7-c643cae73441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:29.374843 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bddfc-0546-4ca7-b3ca-06880b8c91da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:29.376621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bddfc-0546-4ca7-b3ca-06880b8c91da service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:29.380707 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bddfc-0546-4ca7-b3ca-06880b8c91da service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:29.382611 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bddfc-0546-4ca7-b3ca-06880b8c91da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:29.390643 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-358bddfc-0546-4ca7-b3ca-06880b8c91da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:29.460518 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:29.547205 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695c407-e617-4a1b-a9e7-c643cae73441 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:29.565050 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37904476-8635-4be2-9833-8cbd5741878a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:29.570739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37904476-8635-4be2-9833-8cbd5741878a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:29.571340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37904476-8635-4be2-9833-8cbd5741878a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:29.571435 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37904476-8635-4be2-9833-8cbd5741878a admin admin] PATH_INFO: `/v3/users/11a77abcb00e4050a19957e2de2734a4` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:29.573246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37904476-8635-4be2-9833-8cbd5741878a admin admin] RBAC: Authorizing `identity:delete_user(user_id=11a77abcb00e4050a19957e2de2734a4)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:29.575866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37904476-8635-4be2-9833-8cbd5741878a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:29.578991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:29.728845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358bddfc-0546-4ca7-b3ca-06880b8c91da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:29.732003 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358bddfc-0546-4ca7-b3ca-06880b8c91da service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:29.771387 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-37904476-8635-4be2-9833-8cbd5741878a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11a77abcb00e4050a19957e2de2734a4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:29.788114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 962/1870] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 838 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:29.795441 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-37904476-8635-4be2-9833-8cbd5741878a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11a77abcb00e4050a19957e2de2734a4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:29.830036 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-37904476-8635-4be2-9833-8cbd5741878a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11a77abcb00e4050a19957e2de2734a4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:29.830783 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-37904476-8635-4be2-9833-8cbd5741878a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11a77abcb00e4050a19957e2de2734a4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:29.864729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-37904476-8635-4be2-9833-8cbd5741878a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11a77abcb00e4050a19957e2de2734a4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:29.880511 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 909/1871] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:29 2022] DELETE /identity/v3/users/11a77abcb00e4050a19957e2de2734a4 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:30.987610 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bddfc-0546-4ca7-b3ca-06880b8c91da service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:31.003028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bddfc-0546-4ca7-b3ca-06880b8c91da service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:31.165357 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230d50e-34ab-4a10-8bc6-30096a60b8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:31.166109 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230d50e-34ab-4a10-8bc6-30096a60b8c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:31.166439 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230d50e-34ab-4a10-8bc6-30096a60b8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:31.172248 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230d50e-34ab-4a10-8bc6-30096a60b8c9 admin admin] PATH_INFO: `/v3/projects/84cfffbe1f374cafa4f9925cc14cd256` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:31.174206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1230d50e-34ab-4a10-8bc6-30096a60b8c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84cfffbe1f374cafa4f9925cc14cd256)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:31.179780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1230d50e-34ab-4a10-8bc6-30096a60b8c9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:31.337931 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1230d50e-34ab-4a10-8bc6-30096a60b8c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84cfffbe1f374cafa4f9925cc14cd256', 'request_id': 'req-1230d50e-34ab-4a10-8bc6-30096a60b8c9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:31.357551 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1230d50e-34ab-4a10-8bc6-30096a60b8c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84cfffbe1f374cafa4f9925cc14cd256', 'request_id': 'req-1230d50e-34ab-4a10-8bc6-30096a60b8c9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:31.378622 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1230d50e-34ab-4a10-8bc6-30096a60b8c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84cfffbe1f374cafa4f9925cc14cd256', 'request_id': 'req-1230d50e-34ab-4a10-8bc6-30096a60b8c9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:31.407240 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1230d50e-34ab-4a10-8bc6-30096a60b8c9 admin admin] The token cache is being invalidate because project 84cfffbe1f374cafa4f9925cc14cd256 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:31.408185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1230d50e-34ab-4a10-8bc6-30096a60b8c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:31.410724 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1230d50e-34ab-4a10-8bc6-30096a60b8c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:31.414347 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 963/1872] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:30 2022] DELETE /identity/v3/projects/84cfffbe1f374cafa4f9925cc14cd256 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:31.419839 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b42dc-3b1c-4dc7-bb6c-293f5c85ee75 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:31.420512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b42dc-3b1c-4dc7-bb6c-293f5c85ee75 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:31.421409 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b42dc-3b1c-4dc7-bb6c-293f5c85ee75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:31.507954 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-932b42dc-3b1c-4dc7-bb6c-293f5c85ee75 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:31.513431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-932b42dc-3b1c-4dc7-bb6c-293f5c85ee75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:31.702742 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 910/1873] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:31.712373 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ce8b8-f5cb-4c65-923b-6908f4f387e0 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:31.712759 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ce8b8-f5cb-4c65-923b-6908f4f387e0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:31.712993 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ce8b8-f5cb-4c65-923b-6908f4f387e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:31.749080 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d77ce8b8-f5cb-4c65-923b-6908f4f387e0 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:31.750430 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d77ce8b8-f5cb-4c65-923b-6908f4f387e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:31.789876 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 964/1874] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:31.791694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b42dc-3b1c-4dc7-bb6c-293f5c85ee75 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:31.813411 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b42dc-3b1c-4dc7-bb6c-293f5c85ee75 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:31.824313 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a51f1-0317-4d5b-a525-f7fe37fe4b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:31.825019 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a51f1-0317-4d5b-a525-f7fe37fe4b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:31.825361 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a51f1-0317-4d5b-a525-f7fe37fe4b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:31.825856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a51f1-0317-4d5b-a525-f7fe37fe4b0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:31.826592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a51f1-0317-4d5b-a525-f7fe37fe4b0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:33:31.826972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a51f1-0317-4d5b-a525-f7fe37fe4b0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:31.829497 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a51f1-0317-4d5b-a525-f7fe37fe4b0d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:31.850965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ce8b8-f5cb-4c65-923b-6908f4f387e0 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:31.857147 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 911/1875] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:33:31.862007 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ce8b8-f5cb-4c65-923b-6908f4f387e0 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:31.868041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c468077-162f-4bf0-b157-ab53f223d73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:31.868755 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c468077-162f-4bf0-b157-ab53f223d73f admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:31.869099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c468077-162f-4bf0-b157-ab53f223d73f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:31.869360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c468077-162f-4bf0-b157-ab53f223d73f admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:31.870569 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c468077-162f-4bf0-b157-ab53f223d73f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:31.872836 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c468077-162f-4bf0-b157-ab53f223d73f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:31.924567 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 965/1876] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:31 2022] POST /identity/v3/projects => generated 407 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:31.927164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a51f1-0317-4d5b-a525-f7fe37fe4b0d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:31.942643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a51f1-0317-4d5b-a525-f7fe37fe4b0d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:32.055374 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10605d36-2d9f-4054-af7d-d00592a5a9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:32.056096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10605d36-2d9f-4054-af7d-d00592a5a9a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:32.056412 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10605d36-2d9f-4054-af7d-d00592a5a9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:32.057747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10605d36-2d9f-4054-af7d-d00592a5a9a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:32.057747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10605d36-2d9f-4054-af7d-d00592a5a9a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:32.062028 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10605d36-2d9f-4054-af7d-d00592a5a9a7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:32.074243 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:32.094456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 912/1877] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:31 2022] POST /identity/v3/users => generated 304 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:32.109204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c468077-162f-4bf0-b157-ab53f223d73f admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:32.122667 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10605d36-2d9f-4054-af7d-d00592a5a9a7 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:32.124424 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-10605d36-2d9f-4054-af7d-d00592a5a9a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:32.125228 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c468077-162f-4bf0-b157-ab53f223d73f admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:32.130646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e6732-4ceb-4a98-a8a1-dbe3b4e78804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:32.131287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e6732-4ceb-4a98-a8a1-dbe3b4e78804 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:32.131779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e6732-4ceb-4a98-a8a1-dbe3b4e78804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:32.132099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e6732-4ceb-4a98-a8a1-dbe3b4e78804 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:32.132770 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e6732-4ceb-4a98-a8a1-dbe3b4e78804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:32.134527 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377e6732-4ceb-4a98-a8a1-dbe3b4e78804 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:32.149823 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 966/1878] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:32.154706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e6732-4ceb-4a98-a8a1-dbe3b4e78804 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:32.156293 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10605d36-2d9f-4054-af7d-d00592a5a9a7 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:32.163270 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f4b61-23fc-432f-852e-72e19ac2de95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:32.164038 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f4b61-23fc-432f-852e-72e19ac2de95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:32.164382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f4b61-23fc-432f-852e-72e19ac2de95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:32.164727 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f4b61-23fc-432f-852e-72e19ac2de95 admin admin] PATH_INFO: `/v3/users/a5e82b4ae7f547aeb074b3b702d78ccc` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:32.166010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f4b61-23fc-432f-852e-72e19ac2de95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5e82b4ae7f547aeb074b3b702d78ccc)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:32.168345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f4b61-23fc-432f-852e-72e19ac2de95 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:32.169517 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:32.185466 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377e6732-4ceb-4a98-a8a1-dbe3b4e78804 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:32.213123 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54517d-a7fe-4e79-b08f-c324f0344fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:32.216495 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54517d-a7fe-4e79-b08f-c324f0344fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:32.216694 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54517d-a7fe-4e79-b08f-c324f0344fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:32.216996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54517d-a7fe-4e79-b08f-c324f0344fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:32.217699 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54517d-a7fe-4e79-b08f-c324f0344fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:32.219516 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54517d-a7fe-4e79-b08f-c324f0344fb4 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:32.256038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 967/1879] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:32.259840 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54517d-a7fe-4e79-b08f-c324f0344fb4 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:32.262627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4b4f4b61-23fc-432f-852e-72e19ac2de95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5e82b4ae7f547aeb074b3b702d78ccc'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:32.263814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6f54517d-a7fe-4e79-b08f-c324f0344fb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:32.293326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4b4f4b61-23fc-432f-852e-72e19ac2de95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5e82b4ae7f547aeb074b3b702d78ccc'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:32.301405 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4b4f4b61-23fc-432f-852e-72e19ac2de95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5e82b4ae7f547aeb074b3b702d78ccc'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:32.301591 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4b4f4b61-23fc-432f-852e-72e19ac2de95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5e82b4ae7f547aeb074b3b702d78ccc'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:32.321969 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54517d-a7fe-4e79-b08f-c324f0344fb4 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:32.327754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4b4f4b61-23fc-432f-852e-72e19ac2de95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5e82b4ae7f547aeb074b3b702d78ccc'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:32.349822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfbc84e-65c6-4fc7-b343-5fae8c2d2269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:32.349822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbc84e-65c6-4fc7-b343-5fae8c2d2269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:32.349822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbc84e-65c6-4fc7-b343-5fae8c2d2269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:32.349822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfbc84e-65c6-4fc7-b343-5fae8c2d2269 admin admin] PATH_INFO: `/v3/projects/908ee0fecf5a4a078fddabb7e8179fe0` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:32.353642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 913/1880] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:32 2022] DELETE /identity/v3/users/a5e82b4ae7f547aeb074b3b702d78ccc => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:32.354687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfbc84e-65c6-4fc7-b343-5fae8c2d2269 admin admin] RBAC: Authorizing `identity:delete_project(project_id=908ee0fecf5a4a078fddabb7e8179fe0)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:32.356565 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfbc84e-65c6-4fc7-b343-5fae8c2d2269 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:32.357017 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f4b61-23fc-432f-852e-72e19ac2de95 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:32.359357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4b4f4b61-23fc-432f-852e-72e19ac2de95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:32.463448 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-bcfbc84e-65c6-4fc7-b343-5fae8c2d2269 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '908ee0fecf5a4a078fddabb7e8179fe0', 'request_id': 'req-bcfbc84e-65c6-4fc7-b343-5fae8c2d2269'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:32.472078 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f4b61-23fc-432f-852e-72e19ac2de95 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:32.476597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-bcfbc84e-65c6-4fc7-b343-5fae8c2d2269 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '908ee0fecf5a4a078fddabb7e8179fe0', 'request_id': 'req-bcfbc84e-65c6-4fc7-b343-5fae8c2d2269'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:32.488053 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-bcfbc84e-65c6-4fc7-b343-5fae8c2d2269 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '908ee0fecf5a4a078fddabb7e8179fe0', 'request_id': 'req-bcfbc84e-65c6-4fc7-b343-5fae8c2d2269'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:32.509546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-bcfbc84e-65c6-4fc7-b343-5fae8c2d2269 admin admin] The token cache is being invalidate because project 908ee0fecf5a4a078fddabb7e8179fe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:32.510038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-bcfbc84e-65c6-4fc7-b343-5fae8c2d2269 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:32.510929 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-bcfbc84e-65c6-4fc7-b343-5fae8c2d2269 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:32.516852 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 968/1881] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:32 2022] DELETE /identity/v3/projects/908ee0fecf5a4a078fddabb7e8179fe0 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:32.520609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ae443-a1b9-4bed-ab82-ad4ca41666bb None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:32.521638 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ae443-a1b9-4bed-ab82-ad4ca41666bb None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:32.522232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ae443-a1b9-4bed-ab82-ad4ca41666bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:32.552701 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318097d8-ae9c-4647-8db0-a8c41881eb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:32.553387 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318097d8-ae9c-4647-8db0-a8c41881eb6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:32.553668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318097d8-ae9c-4647-8db0-a8c41881eb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:32.553946 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318097d8-ae9c-4647-8db0-a8c41881eb6d admin admin] PATH_INFO: `/v3/projects/b075f5d497314bef9a9ebf2fd5d5a1d2/users/1d38175008134363be61247e645ebe70/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:32.570100 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-0e2ae443-a1b9-4bed-ab82-ad4ca41666bb None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:32.572865 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318097d8-ae9c-4647-8db0-a8c41881eb6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b075f5d497314bef9a9ebf2fd5d5a1d2, user_id=1d38175008134363be61247e645ebe70, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:32.574791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0e2ae443-a1b9-4bed-ab82-ad4ca41666bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:32.575671 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318097d8-ae9c-4647-8db0-a8c41881eb6d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:32.594101 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:32.610042 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 914/1882] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:32 2022] PUT /identity/v3/projects/b075f5d497314bef9a9ebf2fd5d5a1d2/users/1d38175008134363be61247e645ebe70/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:32.611474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318097d8-ae9c-4647-8db0-a8c41881eb6d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:32.615441 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-318097d8-ae9c-4647-8db0-a8c41881eb6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:32.631974 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 969/1883] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:32.631974 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f434c5-cc9c-4d11-a242-fe33becd85bd None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:32.631974 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f434c5-cc9c-4d11-a242-fe33becd85bd None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:32.631974 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f434c5-cc9c-4d11-a242-fe33becd85bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:32.688083 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-47f434c5-cc9c-4d11-a242-fe33becd85bd None None] MFA Rules not processed for user `1d38175008134363be61247e645ebe70`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:32.689671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-47f434c5-cc9c-4d11-a242-fe33becd85bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:32.778451 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318097d8-ae9c-4647-8db0-a8c41881eb6d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:32.791843 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf5a11-90e9-4b27-a248-063f7e857905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:32.792921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf5a11-90e9-4b27-a248-063f7e857905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:32.793455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf5a11-90e9-4b27-a248-063f7e857905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:32.793861 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf5a11-90e9-4b27-a248-063f7e857905 admin admin] PATH_INFO: `/v3/users/f79ca9c2a9274d668ac5a67ae04890f0` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:32.795730 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf5a11-90e9-4b27-a248-063f7e857905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f79ca9c2a9274d668ac5a67ae04890f0)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:32.797424 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 970/1884] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:32 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:32.798885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f434c5-cc9c-4d11-a242-fe33becd85bd None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:32.800550 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-47f434c5-cc9c-4d11-a242-fe33becd85bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:32.803497 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf5a11-90e9-4b27-a248-063f7e857905 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:32.886601 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2acf5a11-90e9-4b27-a248-063f7e857905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f79ca9c2a9274d668ac5a67ae04890f0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:32.894854 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f434c5-cc9c-4d11-a242-fe33becd85bd None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:32.896986 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2acf5a11-90e9-4b27-a248-063f7e857905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f79ca9c2a9274d668ac5a67ae04890f0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:32.899602 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb17257-a93e-44d8-930f-b55f784b5290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:32.900316 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb17257-a93e-44d8-930f-b55f784b5290 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:32.900603 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb17257-a93e-44d8-930f-b55f784b5290 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:32.901025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb17257-a93e-44d8-930f-b55f784b5290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:32.907576 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2acf5a11-90e9-4b27-a248-063f7e857905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f79ca9c2a9274d668ac5a67ae04890f0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:32.908431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2acf5a11-90e9-4b27-a248-063f7e857905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f79ca9c2a9274d668ac5a67ae04890f0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:32.919026 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb17257-a93e-44d8-930f-b55f784b5290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:32.920630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb17257-a93e-44d8-930f-b55f784b5290 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:32.923535 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2acf5a11-90e9-4b27-a248-063f7e857905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f79ca9c2a9274d668ac5a67ae04890f0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:32.931829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:32.937986 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 915/1885] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:32 2022] DELETE /identity/v3/users/f79ca9c2a9274d668ac5a67ae04890f0 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:32.938795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf5a11-90e9-4b27-a248-063f7e857905 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:32.940419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2acf5a11-90e9-4b27-a248-063f7e857905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:32.943491 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 971/1886] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:33.026865 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf5a11-90e9-4b27-a248-063f7e857905 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:33.034549 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b90e9d-fa5f-4c84-aa5e-1f38f89d5bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:33.035414 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b90e9d-fa5f-4c84-aa5e-1f38f89d5bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:33.035658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b90e9d-fa5f-4c84-aa5e-1f38f89d5bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:33.036014 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b90e9d-fa5f-4c84-aa5e-1f38f89d5bb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:33.036732 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b90e9d-fa5f-4c84-aa5e-1f38f89d5bb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:33.038826 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b90e9d-fa5f-4c84-aa5e-1f38f89d5bb4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:33.073171 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 916/1887] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:32 2022] POST /identity/v3/projects => generated 409 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:33.078107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb17257-a93e-44d8-930f-b55f784b5290 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:33.081940 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-dbb17257-a93e-44d8-930f-b55f784b5290 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:33.109240 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb17257-a93e-44d8-930f-b55f784b5290 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:33.224233 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef122a2-c590-4e4d-b3c8-d3ddc1d4ba81 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:33.224926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef122a2-c590-4e4d-b3c8-d3ddc1d4ba81 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:33.231175 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef122a2-c590-4e4d-b3c8-d3ddc1d4ba81 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:33.234094 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 917/1888] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 29 20:33:33 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:33:33.255233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6feac09-4866-4534-b102-fcdb97a39680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:33.256703 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6feac09-4866-4534-b102-fcdb97a39680 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:33.257364 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6feac09-4866-4534-b102-fcdb97a39680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:33.259219 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6feac09-4866-4534-b102-fcdb97a39680 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:33.260802 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6feac09-4866-4534-b102-fcdb97a39680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:33.264775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6feac09-4866-4534-b102-fcdb97a39680 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:33.264909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fa7a5-6985-4e84-8619-2106144cdb77 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:33.274104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fa7a5-6985-4e84-8619-2106144cdb77 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:33.274472 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fa7a5-6985-4e84-8619-2106144cdb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:33.296112 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:33.318295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c89fa7a5-6985-4e84-8619-2106144cdb77 None None] MFA Rules not processed for user `c0c44754a1c04a2b87081aba6cb509d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:33.319677 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c89fa7a5-6985-4e84-8619-2106144cdb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:33.351530 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 972/1889] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:33 2022] POST /identity/v3/users => generated 311 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:33.353023 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6feac09-4866-4534-b102-fcdb97a39680 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:33.381143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6feac09-4866-4534-b102-fcdb97a39680 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:33.395275 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6995bc-5bae-4966-a424-d77add8ce97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:33.396288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6995bc-5bae-4966-a424-d77add8ce97d admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:33.396960 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6995bc-5bae-4966-a424-d77add8ce97d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:33.397401 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6995bc-5bae-4966-a424-d77add8ce97d admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:33.398649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6995bc-5bae-4966-a424-d77add8ce97d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:33.403330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6995bc-5bae-4966-a424-d77add8ce97d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:33.436227 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 973/1890] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:33.439228 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6995bc-5bae-4966-a424-d77add8ce97d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:33.462705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6995bc-5bae-4966-a424-d77add8ce97d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:33.480642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb163bfa-ab5a-4b3c-93e5-adbaa7925236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:33.482163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb163bfa-ab5a-4b3c-93e5-adbaa7925236 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:33.482586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb163bfa-ab5a-4b3c-93e5-adbaa7925236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:33.483199 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb163bfa-ab5a-4b3c-93e5-adbaa7925236 admin admin] PATH_INFO: `/v3/projects/e2ad6bacd0a94d66ab4285164b912fa2/users/7a2df26e6e9e418e923a5b3bfe344b73/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:33.493954 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 918/1891] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 29 20:33:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:33.526470 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fa7a5-6985-4e84-8619-2106144cdb77 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:33.528762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c89fa7a5-6985-4e84-8619-2106144cdb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:33.535488 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb163bfa-ab5a-4b3c-93e5-adbaa7925236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2ad6bacd0a94d66ab4285164b912fa2, user_id=7a2df26e6e9e418e923a5b3bfe344b73, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:33.545953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb163bfa-ab5a-4b3c-93e5-adbaa7925236 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:33.556581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fa7a5-6985-4e84-8619-2106144cdb77 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:33.571366 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5fad1-613c-46c6-869f-a24695bd997c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:33.572186 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5fad1-613c-46c6-869f-a24695bd997c service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:33.572592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5fad1-613c-46c6-869f-a24695bd997c service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:33.572944 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5fad1-613c-46c6-869f-a24695bd997c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:33.586941 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5fad1-613c-46c6-869f-a24695bd997c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:33.589889 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5fad1-613c-46c6-869f-a24695bd997c service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:33.595241 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 974/1892] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:33 2022] PUT /identity/v3/projects/e2ad6bacd0a94d66ab4285164b912fa2/users/7a2df26e6e9e418e923a5b3bfe344b73/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:33.602150 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb163bfa-ab5a-4b3c-93e5-adbaa7925236 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:33.605941 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-eb163bfa-ab5a-4b3c-93e5-adbaa7925236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:33.613722 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 919/1893] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:33.614826 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5fad1-613c-46c6-869f-a24695bd997c service nova] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:33.646532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5fad1-613c-46c6-869f-a24695bd997c service nova] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:33.844845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb163bfa-ab5a-4b3c-93e5-adbaa7925236 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:33.856979 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64439bef-7845-4c1a-885a-07ba40b13d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:33.859787 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64439bef-7845-4c1a-885a-07ba40b13d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:33.860201 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64439bef-7845-4c1a-885a-07ba40b13d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:33.861852 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64439bef-7845-4c1a-885a-07ba40b13d0c admin admin] PATH_INFO: `/v3/projects/eb689752adfe4c92aaa6e5e9bbf25a6b` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:33.869636 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64439bef-7845-4c1a-885a-07ba40b13d0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb689752adfe4c92aaa6e5e9bbf25a6b)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:33.874038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64439bef-7845-4c1a-885a-07ba40b13d0c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:33.881982 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137cdcf-fe0d-4efe-8930-fad0de6fab99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:33.884121 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137cdcf-fe0d-4efe-8930-fad0de6fab99 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:33.884501 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137cdcf-fe0d-4efe-8930-fad0de6fab99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:33.887376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137cdcf-fe0d-4efe-8930-fad0de6fab99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:33.887376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137cdcf-fe0d-4efe-8930-fad0de6fab99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:33.887599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137cdcf-fe0d-4efe-8930-fad0de6fab99 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:33.913530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 920/1894] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:33.919252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137cdcf-fe0d-4efe-8930-fad0de6fab99 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:33.921184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2137cdcf-fe0d-4efe-8930-fad0de6fab99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:33.935431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137cdcf-fe0d-4efe-8930-fad0de6fab99 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:33.942516 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fcd80-3b8d-4126-90f1-674e6c0fa54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:33.943202 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fcd80-3b8d-4126-90f1-674e6c0fa54d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:33.943622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fcd80-3b8d-4126-90f1-674e6c0fa54d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:33.943882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fcd80-3b8d-4126-90f1-674e6c0fa54d admin admin] PATH_INFO: `/v3/projects/1a8cdc6e922348f490c740427062c261` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:33.960727 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fcd80-3b8d-4126-90f1-674e6c0fa54d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a8cdc6e922348f490c740427062c261)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:33.964089 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fcd80-3b8d-4126-90f1-674e6c0fa54d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:33.969541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-64439bef-7845-4c1a-885a-07ba40b13d0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb689752adfe4c92aaa6e5e9bbf25a6b', 'request_id': 'req-64439bef-7845-4c1a-885a-07ba40b13d0c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:33.981104 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-64439bef-7845-4c1a-885a-07ba40b13d0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb689752adfe4c92aaa6e5e9bbf25a6b', 'request_id': 'req-64439bef-7845-4c1a-885a-07ba40b13d0c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:33.984587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:34.000759 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-64439bef-7845-4c1a-885a-07ba40b13d0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb689752adfe4c92aaa6e5e9bbf25a6b', 'request_id': 'req-64439bef-7845-4c1a-885a-07ba40b13d0c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.017245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-64439bef-7845-4c1a-885a-07ba40b13d0c admin admin] The token cache is being invalidate because project eb689752adfe4c92aaa6e5e9bbf25a6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:34.017608 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-64439bef-7845-4c1a-885a-07ba40b13d0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.018840 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-64439bef-7845-4c1a-885a-07ba40b13d0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.021756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 975/1895] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:33 2022] DELETE /identity/v3/projects/eb689752adfe4c92aaa6e5e9bbf25a6b => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:34.025018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64439bef-7845-4c1a-885a-07ba40b13d0c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:34.025018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-64439bef-7845-4c1a-885a-07ba40b13d0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:34.067096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d39fcd80-3b8d-4126-90f1-674e6c0fa54d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a8cdc6e922348f490c740427062c261', 'request_id': 'req-d39fcd80-3b8d-4126-90f1-674e6c0fa54d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.075589 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d39fcd80-3b8d-4126-90f1-674e6c0fa54d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a8cdc6e922348f490c740427062c261', 'request_id': 'req-d39fcd80-3b8d-4126-90f1-674e6c0fa54d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.096788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d39fcd80-3b8d-4126-90f1-674e6c0fa54d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a8cdc6e922348f490c740427062c261', 'request_id': 'req-d39fcd80-3b8d-4126-90f1-674e6c0fa54d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.110977 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:34.119621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d39fcd80-3b8d-4126-90f1-674e6c0fa54d admin admin] The token cache is being invalidate because project 1a8cdc6e922348f490c740427062c261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:34.119945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d39fcd80-3b8d-4126-90f1-674e6c0fa54d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.121144 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d39fcd80-3b8d-4126-90f1-674e6c0fa54d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.142882 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64439bef-7845-4c1a-885a-07ba40b13d0c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:34.146208 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 921/1896] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:33 2022] DELETE /identity/v3/projects/1a8cdc6e922348f490c740427062c261 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:34.152436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fcd80-3b8d-4126-90f1-674e6c0fa54d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:34.157341 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d39fcd80-3b8d-4126-90f1-674e6c0fa54d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:34.258592 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc0d2e-763e-4e7e-a711-9318b79f1a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:34.259333 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc0d2e-763e-4e7e-a711-9318b79f1a91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:34.259628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc0d2e-763e-4e7e-a711-9318b79f1a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:34.259901 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc0d2e-763e-4e7e-a711-9318b79f1a91 admin admin] PATH_INFO: `/v3/projects/e2ad6bacd0a94d66ab4285164b912fa2/users/7a2df26e6e9e418e923a5b3bfe344b73/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:34.261491 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fcd80-3b8d-4126-90f1-674e6c0fa54d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:34.263028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc0d2e-763e-4e7e-a711-9318b79f1a91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2ad6bacd0a94d66ab4285164b912fa2, user_id=7a2df26e6e9e418e923a5b3bfe344b73, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:34.267333 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc0d2e-763e-4e7e-a711-9318b79f1a91 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:34.267541 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cdd22b-d6b6-4bd7-ae62-c984524841e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:34.268252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdd22b-d6b6-4bd7-ae62-c984524841e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:34.268541 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdd22b-d6b6-4bd7-ae62-c984524841e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:34.268818 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cdd22b-d6b6-4bd7-ae62-c984524841e8 admin admin] PATH_INFO: `/v3/users/5245be497c1a4cf9a32e33ed54727442` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:34.270972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cdd22b-d6b6-4bd7-ae62-c984524841e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5245be497c1a4cf9a32e33ed54727442)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:34.273540 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cdd22b-d6b6-4bd7-ae62-c984524841e8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:34.282734 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 976/1897] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:34 2022] PUT /identity/v3/projects/e2ad6bacd0a94d66ab4285164b912fa2/users/7a2df26e6e9e418e923a5b3bfe344b73/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:34.283379 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc0d2e-763e-4e7e-a711-9318b79f1a91 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:34.295824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5ddc0d2e-763e-4e7e-a711-9318b79f1a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:34.339462 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-32cdd22b-d6b6-4bd7-ae62-c984524841e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5245be497c1a4cf9a32e33ed54727442'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.348974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-32cdd22b-d6b6-4bd7-ae62-c984524841e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5245be497c1a4cf9a32e33ed54727442'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.360078 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-32cdd22b-d6b6-4bd7-ae62-c984524841e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5245be497c1a4cf9a32e33ed54727442'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.361107 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-32cdd22b-d6b6-4bd7-ae62-c984524841e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5245be497c1a4cf9a32e33ed54727442'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.377679 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-32cdd22b-d6b6-4bd7-ae62-c984524841e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5245be497c1a4cf9a32e33ed54727442'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.389537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 922/1898] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:34 2022] DELETE /identity/v3/users/5245be497c1a4cf9a32e33ed54727442 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:34.391673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed15b37-e24e-4da0-be64-a390d9c4e486 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:34.391998 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed15b37-e24e-4da0-be64-a390d9c4e486 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:34.392279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed15b37-e24e-4da0-be64-a390d9c4e486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:34.415495 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc0d2e-763e-4e7e-a711-9318b79f1a91 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:34.449839 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-bed15b37-e24e-4da0-be64-a390d9c4e486 None None] MFA Rules not processed for user `7a2df26e6e9e418e923a5b3bfe344b73`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:34.452339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bed15b37-e24e-4da0-be64-a390d9c4e486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:34.471220 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:34.514789 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874dd4d-24ea-41be-aa89-961b90c22854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:34.515948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874dd4d-24ea-41be-aa89-961b90c22854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:34.516407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874dd4d-24ea-41be-aa89-961b90c22854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:34.516836 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874dd4d-24ea-41be-aa89-961b90c22854 admin admin] PATH_INFO: `/v3/users/fba92dfe70094c9c8a9b94bcd2cd7309` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:34.518571 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874dd4d-24ea-41be-aa89-961b90c22854 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fba92dfe70094c9c8a9b94bcd2cd7309)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:34.521562 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874dd4d-24ea-41be-aa89-961b90c22854 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:34.563488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 923/1899] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:34 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:34.577854 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed15b37-e24e-4da0-be64-a390d9c4e486 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:34.579275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bed15b37-e24e-4da0-be64-a390d9c4e486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:34.586918 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:34.607620 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a874dd4d-24ea-41be-aa89-961b90c22854 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fba92dfe70094c9c8a9b94bcd2cd7309'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.618305 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a874dd4d-24ea-41be-aa89-961b90c22854 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fba92dfe70094c9c8a9b94bcd2cd7309'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.631332 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a874dd4d-24ea-41be-aa89-961b90c22854 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fba92dfe70094c9c8a9b94bcd2cd7309'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.633115 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a874dd4d-24ea-41be-aa89-961b90c22854 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fba92dfe70094c9c8a9b94bcd2cd7309'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.634037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a874dd4d-24ea-41be-aa89-961b90c22854 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fba92dfe70094c9c8a9b94bcd2cd7309'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.663814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 977/1900] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:34 2022] DELETE /identity/v3/users/fba92dfe70094c9c8a9b94bcd2cd7309 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:34.687800 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed15b37-e24e-4da0-be64-a390d9c4e486 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:34.759678 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7abd2a-2f6a-4136-bebb-735a0a6c9f73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:34.760504 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7abd2a-2f6a-4136-bebb-735a0a6c9f73 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:34.760933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7abd2a-2f6a-4136-bebb-735a0a6c9f73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:34.761252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7abd2a-2f6a-4136-bebb-735a0a6c9f73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:34.763174 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9b7abd2a-2f6a-4136-bebb-735a0a6c9f73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:34.774661 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874dd4d-24ea-41be-aa89-961b90c22854 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:34.777295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a874dd4d-24ea-41be-aa89-961b90c22854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:34.849250 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7abd2a-2f6a-4136-bebb-735a0a6c9f73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:34.850622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7abd2a-2f6a-4136-bebb-735a0a6c9f73 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:34.863222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 924/1901] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:34 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:33:34.893091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874dd4d-24ea-41be-aa89-961b90c22854 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:34.898710 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52142251-7f01-4361-9d38-0541d015b90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:34.899526 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52142251-7f01-4361-9d38-0541d015b90d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:34.899732 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52142251-7f01-4361-9d38-0541d015b90d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:34.900001 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52142251-7f01-4361-9d38-0541d015b90d admin admin] PATH_INFO: `/v3/projects/57d5176d914d4981ba9042a76aacf2c3` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:34.901267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52142251-7f01-4361-9d38-0541d015b90d admin admin] RBAC: Authorizing `identity:delete_project(project_id=57d5176d914d4981ba9042a76aacf2c3)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:34.903286 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52142251-7f01-4361-9d38-0541d015b90d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:34.964696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-52142251-7f01-4361-9d38-0541d015b90d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57d5176d914d4981ba9042a76aacf2c3', 'request_id': 'req-52142251-7f01-4361-9d38-0541d015b90d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.972651 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:34.980666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-52142251-7f01-4361-9d38-0541d015b90d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57d5176d914d4981ba9042a76aacf2c3', 'request_id': 'req-52142251-7f01-4361-9d38-0541d015b90d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:34.992405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-52142251-7f01-4361-9d38-0541d015b90d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57d5176d914d4981ba9042a76aacf2c3', 'request_id': 'req-52142251-7f01-4361-9d38-0541d015b90d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:35.015379 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-52142251-7f01-4361-9d38-0541d015b90d admin admin] The token cache is being invalidate because project 57d5176d914d4981ba9042a76aacf2c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:35.015379 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-52142251-7f01-4361-9d38-0541d015b90d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:35.016774 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-52142251-7f01-4361-9d38-0541d015b90d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:35.018527 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 978/1902] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:34 2022] DELETE /identity/v3/projects/57d5176d914d4981ba9042a76aacf2c3 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:35.025061 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a406f-b13f-4c70-8287-eac60bd72515 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:35.025445 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a406f-b13f-4c70-8287-eac60bd72515 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:35.025796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a406f-b13f-4c70-8287-eac60bd72515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:35.054508 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-7f2a406f-b13f-4c70-8287-eac60bd72515 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:35.055795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7f2a406f-b13f-4c70-8287-eac60bd72515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:35.110809 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52142251-7f01-4361-9d38-0541d015b90d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:35.113055 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-52142251-7f01-4361-9d38-0541d015b90d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:35.227333 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 925/1903] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:35.229377 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52142251-7f01-4361-9d38-0541d015b90d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:35.229878 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4eecfd-2cd0-4dfe-9895-69b91e5ba0e7 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:35.230023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4eecfd-2cd0-4dfe-9895-69b91e5ba0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:35.230365 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4eecfd-2cd0-4dfe-9895-69b91e5ba0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:35.242960 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdceb5e-ed32-41bd-92e6-d5875833849f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:35.244107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdceb5e-ed32-41bd-92e6-d5875833849f service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:35.244602 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdceb5e-ed32-41bd-92e6-d5875833849f service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:35.245071 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdceb5e-ed32-41bd-92e6-d5875833849f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:35.248875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-bcdceb5e-ed32-41bd-92e6-d5875833849f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:35.277042 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ea4eecfd-2cd0-4dfe-9895-69b91e5ba0e7 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:35.278840 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ea4eecfd-2cd0-4dfe-9895-69b91e5ba0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:35.316089 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 926/1904] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:35.317620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4eecfd-2cd0-4dfe-9895-69b91e5ba0e7 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:35.332766 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:35.342040 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4eecfd-2cd0-4dfe-9895-69b91e5ba0e7 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:35.354248 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d045b-9d38-4c0d-8538-0c08d332c60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:35.354248 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d045b-9d38-4c0d-8538-0c08d332c60c admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:35.354248 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d045b-9d38-4c0d-8538-0c08d332c60c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:35.354248 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d045b-9d38-4c0d-8538-0c08d332c60c admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:35.354783 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d045b-9d38-4c0d-8538-0c08d332c60c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:33:35.355034 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d045b-9d38-4c0d-8538-0c08d332c60c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:35.356714 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d045b-9d38-4c0d-8538-0c08d332c60c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:35.385030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 927/1905] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:33:35.390806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d045b-9d38-4c0d-8538-0c08d332c60c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:35.416912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d045b-9d38-4c0d-8538-0c08d332c60c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:35.427131 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084e53f-a2ad-4a8e-a9c3-969753864d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:35.427890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084e53f-a2ad-4a8e-a9c3-969753864d0c admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:35.428416 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084e53f-a2ad-4a8e-a9c3-969753864d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:35.428688 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084e53f-a2ad-4a8e-a9c3-969753864d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:35.429443 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b084e53f-a2ad-4a8e-a9c3-969753864d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:35.431778 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b084e53f-a2ad-4a8e-a9c3-969753864d0c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:35.446733 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdceb5e-ed32-41bd-92e6-d5875833849f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:35.449589 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdceb5e-ed32-41bd-92e6-d5875833849f service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:35.478317 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 979/1906] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:35 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:33:35.515791 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 928/1907] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:35 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:35.516490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdceb5e-ed32-41bd-92e6-d5875833849f service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:35.533179 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdceb5e-ed32-41bd-92e6-d5875833849f service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:35.626899 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01e48c-a08a-4832-8a61-986c012cf8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:35.629935 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01e48c-a08a-4832-8a61-986c012cf8cc admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:35.630246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01e48c-a08a-4832-8a61-986c012cf8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:35.630965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01e48c-a08a-4832-8a61-986c012cf8cc admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:35.631624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01e48c-a08a-4832-8a61-986c012cf8cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:35.637578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01e48c-a08a-4832-8a61-986c012cf8cc admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:35.683688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 980/1908] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:35 2022] POST /identity/v3/users => generated 292 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:35.687885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084e53f-a2ad-4a8e-a9c3-969753864d0c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:35.699227 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084e53f-a2ad-4a8e-a9c3-969753864d0c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:35.712812 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71f17e-dba0-447c-a671-dd5cc6c9a972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:35.713447 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71f17e-dba0-447c-a671-dd5cc6c9a972 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:35.713730 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71f17e-dba0-447c-a671-dd5cc6c9a972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:35.714004 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71f17e-dba0-447c-a671-dd5cc6c9a972 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:35.714668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71f17e-dba0-447c-a671-dd5cc6c9a972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:35.716385 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71f17e-dba0-447c-a671-dd5cc6c9a972 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:35.742094 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 929/1909] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:35.746998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01e48c-a08a-4832-8a61-986c012cf8cc admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:35.766607 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01e48c-a08a-4832-8a61-986c012cf8cc admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:35.790324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded8c898-037b-43b1-bc1c-610ba1733e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:35.792089 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8c898-037b-43b1-bc1c-610ba1733e6e admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:35.795119 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8c898-037b-43b1-bc1c-610ba1733e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:35.795119 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8c898-037b-43b1-bc1c-610ba1733e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:35.797733 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded8c898-037b-43b1-bc1c-610ba1733e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:35.806977 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded8c898-037b-43b1-bc1c-610ba1733e6e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:35.844495 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 981/1910] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:35.850104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71f17e-dba0-447c-a671-dd5cc6c9a972 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:35.866181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71f17e-dba0-447c-a671-dd5cc6c9a972 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:35.878556 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffe1fa-7537-42c4-b150-f32bb55325fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:35.878556 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffe1fa-7537-42c4-b150-f32bb55325fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:35.879000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffe1fa-7537-42c4-b150-f32bb55325fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:35.880204 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaffe1fa-7537-42c4-b150-f32bb55325fc admin admin] PATH_INFO: `/v3/projects/e2f92e90a6fa46fb819389b138a9c0c1/users/a171c3d8442a4000b977d8f484d25d3f/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:35.937917 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaffe1fa-7537-42c4-b150-f32bb55325fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2f92e90a6fa46fb819389b138a9c0c1, user_id=a171c3d8442a4000b977d8f484d25d3f, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:35.945388 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaffe1fa-7537-42c4-b150-f32bb55325fc admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:35.983065 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 930/1911] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:35 2022] PUT /identity/v3/projects/e2f92e90a6fa46fb819389b138a9c0c1/users/a171c3d8442a4000b977d8f484d25d3f/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:35.999788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39928ebf-7047-4631-ab91-ece1f3b5428f None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:36.000580 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39928ebf-7047-4631-ab91-ece1f3b5428f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:36.001040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39928ebf-7047-4631-ab91-ece1f3b5428f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:36.095005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-39928ebf-7047-4631-ab91-ece1f3b5428f None None] MFA Rules not processed for user `a171c3d8442a4000b977d8f484d25d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:36.096331 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-39928ebf-7047-4631-ab91-ece1f3b5428f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:36.101949 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:36.313748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 982/1912] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:36.320421 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffe1fa-7537-42c4-b150-f32bb55325fc admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:36.368531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39928ebf-7047-4631-ab91-ece1f3b5428f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:36.376267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-39928ebf-7047-4631-ab91-ece1f3b5428f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:36.433705 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaffe1fa-7537-42c4-b150-f32bb55325fc admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:36.438914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb19fc-14f8-4edc-94f1-4e6524c454ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:36.439564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb19fc-14f8-4edc-94f1-4e6524c454ce admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:36.439848 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb19fc-14f8-4edc-94f1-4e6524c454ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:36.440170 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb19fc-14f8-4edc-94f1-4e6524c454ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:36.440805 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb19fc-14f8-4edc-94f1-4e6524c454ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:36.443374 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb19fc-14f8-4edc-94f1-4e6524c454ce admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:36.467533 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39928ebf-7047-4631-ab91-ece1f3b5428f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:36.476346 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 931/1913] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:36 2022] POST /identity/v3/projects => generated 381 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:36.476591 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e22e6-775d-493c-aa33-3bd1a091d95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:36.476591 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e22e6-775d-493c-aa33-3bd1a091d95a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:36.476591 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e22e6-775d-493c-aa33-3bd1a091d95a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:36.476591 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e22e6-775d-493c-aa33-3bd1a091d95a admin admin] PATH_INFO: `/v3/projects/3464fe4aaa7543019c06ef1d8edb1e61` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:36.477172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e22e6-775d-493c-aa33-3bd1a091d95a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3464fe4aaa7543019c06ef1d8edb1e61)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:36.477172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e22e6-775d-493c-aa33-3bd1a091d95a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:36.478604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb19fc-14f8-4edc-94f1-4e6524c454ce admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:36.527860 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-2b0e22e6-775d-493c-aa33-3bd1a091d95a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3464fe4aaa7543019c06ef1d8edb1e61', 'request_id': 'req-2b0e22e6-775d-493c-aa33-3bd1a091d95a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:36.535434 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-2b0e22e6-775d-493c-aa33-3bd1a091d95a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3464fe4aaa7543019c06ef1d8edb1e61', 'request_id': 'req-2b0e22e6-775d-493c-aa33-3bd1a091d95a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:36.541083 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-2b0e22e6-775d-493c-aa33-3bd1a091d95a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3464fe4aaa7543019c06ef1d8edb1e61', 'request_id': 'req-2b0e22e6-775d-493c-aa33-3bd1a091d95a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:36.553324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-2b0e22e6-775d-493c-aa33-3bd1a091d95a admin admin] The token cache is being invalidate because project 3464fe4aaa7543019c06ef1d8edb1e61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:36.553621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-2b0e22e6-775d-493c-aa33-3bd1a091d95a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:36.554298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-2b0e22e6-775d-493c-aa33-3bd1a091d95a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:36.555846 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 983/1914] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:36 2022] DELETE /identity/v3/projects/3464fe4aaa7543019c06ef1d8edb1e61 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:36.558235 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb19fc-14f8-4edc-94f1-4e6524c454ce admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:36.560594 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73532a40-ab98-4133-ae07-9b2e96850a69 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:36.560818 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73532a40-ab98-4133-ae07-9b2e96850a69 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:36.561097 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73532a40-ab98-4133-ae07-9b2e96850a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:36.583695 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:36.593542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-73532a40-ab98-4133-ae07-9b2e96850a69 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:36.595628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-73532a40-ab98-4133-ae07-9b2e96850a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:36.617651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9f182-7970-4a29-922c-c4d7b7207a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:36.618307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9f182-7970-4a29-922c-c4d7b7207a9a admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:36.618781 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9f182-7970-4a29-922c-c4d7b7207a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:36.619149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9f182-7970-4a29-922c-c4d7b7207a9a admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:36.619525 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9f182-7970-4a29-922c-c4d7b7207a9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:36.621791 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9f182-7970-4a29-922c-c4d7b7207a9a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:36.639043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:36.646927 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 932/1915] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:36 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:36.653176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9f182-7970-4a29-922c-c4d7b7207a9a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:36.655857 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-abb9f182-7970-4a29-922c-c4d7b7207a9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:36.668735 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9f182-7970-4a29-922c-c4d7b7207a9a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:36.674780 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29291758-e5ed-46b3-b475-6eb274ac8968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:36.675497 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29291758-e5ed-46b3-b475-6eb274ac8968 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:36.675881 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29291758-e5ed-46b3-b475-6eb274ac8968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:36.676795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29291758-e5ed-46b3-b475-6eb274ac8968 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:36.677488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29291758-e5ed-46b3-b475-6eb274ac8968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:36.679495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29291758-e5ed-46b3-b475-6eb274ac8968 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:36.691251 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 984/1916] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:36.691470 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 933/1917] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:36.696195 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29291758-e5ed-46b3-b475-6eb274ac8968 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:36.696928 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e9983-9e49-4743-86b9-8f2eecf6ec80 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:36.697225 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e9983-9e49-4743-86b9-8f2eecf6ec80 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:36.697505 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e9983-9e49-4743-86b9-8f2eecf6ec80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:36.706411 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29291758-e5ed-46b3-b475-6eb274ac8968 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:36.711611 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e957684-9e6c-4125-b949-456d8f0073f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:36.712311 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e957684-9e6c-4125-b949-456d8f0073f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:36.712591 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e957684-9e6c-4125-b949-456d8f0073f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:36.712863 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e957684-9e6c-4125-b949-456d8f0073f0 admin admin] PATH_INFO: `/v3/projects/c534a25c0fca42719e3c316eabd0def8/users/78aed86bc2234c6daf0b8ce40d686e8d/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:36.718999 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-5c8e9983-9e49-4743-86b9-8f2eecf6ec80 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:36.720347 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5c8e9983-9e49-4743-86b9-8f2eecf6ec80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:36.729068 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e957684-9e6c-4125-b949-456d8f0073f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c534a25c0fca42719e3c316eabd0def8, user_id=78aed86bc2234c6daf0b8ce40d686e8d, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:36.730460 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 985/1918] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:36.733091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e957684-9e6c-4125-b949-456d8f0073f0 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:36.735885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e9983-9e49-4743-86b9-8f2eecf6ec80 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:36.747195 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e9983-9e49-4743-86b9-8f2eecf6ec80 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:36.749825 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 934/1919] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:36 2022] PUT /identity/v3/projects/c534a25c0fca42719e3c316eabd0def8/users/78aed86bc2234c6daf0b8ce40d686e8d/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:36.765247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e957684-9e6c-4125-b949-456d8f0073f0 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:36.819592 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81afed2a-986c-4862-94b3-2abd858a025b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:36.820250 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afed2a-986c-4862-94b3-2abd858a025b admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:36.820531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afed2a-986c-4862-94b3-2abd858a025b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:36.820798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afed2a-986c-4862-94b3-2abd858a025b admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:36.821460 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81afed2a-986c-4862-94b3-2abd858a025b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:33:36.821764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81afed2a-986c-4862-94b3-2abd858a025b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:36.823362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81afed2a-986c-4862-94b3-2abd858a025b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:36.831798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e957684-9e6c-4125-b949-456d8f0073f0 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:36.837270 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 986/1920] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:33:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:33:36.838038 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8403888-7525-47e3-8e54-a1f506156ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:36.838655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8403888-7525-47e3-8e54-a1f506156ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:36.838938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8403888-7525-47e3-8e54-a1f506156ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:36.839214 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8403888-7525-47e3-8e54-a1f506156ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:36.839861 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8403888-7525-47e3-8e54-a1f506156ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:36.841465 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81afed2a-986c-4862-94b3-2abd858a025b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:36.841577 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8403888-7525-47e3-8e54-a1f506156ce2 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:36.852132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81afed2a-986c-4862-94b3-2abd858a025b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:36.852703 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 935/1921] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:33:36.856674 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8403888-7525-47e3-8e54-a1f506156ce2 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:36.860650 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b027a-dc45-430f-8bf3-c2cd89620307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:36.861429 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b027a-dc45-430f-8bf3-c2cd89620307 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:36.861709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b027a-dc45-430f-8bf3-c2cd89620307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:36.861984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b027a-dc45-430f-8bf3-c2cd89620307 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:36.862618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b027a-dc45-430f-8bf3-c2cd89620307 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:36.864640 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67b027a-dc45-430f-8bf3-c2cd89620307 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:36.866612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8403888-7525-47e3-8e54-a1f506156ce2 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:36.871203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766e8aa-95d8-4bdd-820f-2b907f1bcdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:36.871988 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766e8aa-95d8-4bdd-820f-2b907f1bcdc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:36.872174 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766e8aa-95d8-4bdd-820f-2b907f1bcdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:36.872448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766e8aa-95d8-4bdd-820f-2b907f1bcdc0 admin admin] PATH_INFO: `/v3/projects/c534a25c0fca42719e3c316eabd0def8/users/78aed86bc2234c6daf0b8ce40d686e8d/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:36.874359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4766e8aa-95d8-4bdd-820f-2b907f1bcdc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c534a25c0fca42719e3c316eabd0def8, user_id=78aed86bc2234c6daf0b8ce40d686e8d, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:36.877119 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4766e8aa-95d8-4bdd-820f-2b907f1bcdc0 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:36.889230 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 936/1922] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:36 2022] PUT /identity/v3/projects/c534a25c0fca42719e3c316eabd0def8/users/78aed86bc2234c6daf0b8ce40d686e8d/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:36.898673 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 987/1923] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:36 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:36.901564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566f353-37b3-4a60-831f-0e7b756b066b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:36.901909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566f353-37b3-4a60-831f-0e7b756b066b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:36.903392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566f353-37b3-4a60-831f-0e7b756b066b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:36.903860 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b027a-dc45-430f-8bf3-c2cd89620307 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:36.932741 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c566f353-37b3-4a60-831f-0e7b756b066b None None] MFA Rules not processed for user `78aed86bc2234c6daf0b8ce40d686e8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:36.934097 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c566f353-37b3-4a60-831f-0e7b756b066b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:36.970733 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b027a-dc45-430f-8bf3-c2cd89620307 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:36.982970 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77342e-3b43-4e78-ae31-555a730c4471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:36.983613 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77342e-3b43-4e78-ae31-555a730c4471 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:36.983899 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77342e-3b43-4e78-ae31-555a730c4471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:36.984202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77342e-3b43-4e78-ae31-555a730c4471 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:36.984849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77342e-3b43-4e78-ae31-555a730c4471 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:36.986766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77342e-3b43-4e78-ae31-555a730c4471 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:37.003588 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:37.025761 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 988/1924] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:36 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:37.029072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 937/1925] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:36 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:37.032309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77342e-3b43-4e78-ae31-555a730c4471 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:37.046557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77342e-3b43-4e78-ae31-555a730c4471 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:37.046791 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566f353-37b3-4a60-831f-0e7b756b066b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:37.049597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c566f353-37b3-4a60-831f-0e7b756b066b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:37.053704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c239d-716a-49be-bde9-f39fe6c1cba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:37.054775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c239d-716a-49be-bde9-f39fe6c1cba2 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:37.055298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c239d-716a-49be-bde9-f39fe6c1cba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:37.055763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c239d-716a-49be-bde9-f39fe6c1cba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:37.057168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c239d-716a-49be-bde9-f39fe6c1cba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:37.059128 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c239d-716a-49be-bde9-f39fe6c1cba2 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:37.073940 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 989/1926] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:37.076993 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c239d-716a-49be-bde9-f39fe6c1cba2 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:37.087508 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c239d-716a-49be-bde9-f39fe6c1cba2 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:37.093326 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ee239-e23e-4601-a33c-4d5cd594af02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:37.093326 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ee239-e23e-4601-a33c-4d5cd594af02 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:37.093326 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ee239-e23e-4601-a33c-4d5cd594af02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:37.093653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ee239-e23e-4601-a33c-4d5cd594af02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:37.094140 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ee239-e23e-4601-a33c-4d5cd594af02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:37.095751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ee239-e23e-4601-a33c-4d5cd594af02 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:37.107843 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 990/1927] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:37.112166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ee239-e23e-4601-a33c-4d5cd594af02 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:37.114515 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566f353-37b3-4a60-831f-0e7b756b066b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:37.121280 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3edc11-ef42-430f-9307-98a333114826 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:37.122137 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3edc11-ef42-430f-9307-98a333114826 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:37.122253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3edc11-ef42-430f-9307-98a333114826 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:37.122435 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3edc11-ef42-430f-9307-98a333114826 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:37.123701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ee239-e23e-4601-a33c-4d5cd594af02 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:37.130262 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfda58f-99fc-4442-96a2-c70e5bfba511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:37.131160 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfda58f-99fc-4442-96a2-c70e5bfba511 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:37.131526 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfda58f-99fc-4442-96a2-c70e5bfba511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:37.131872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfda58f-99fc-4442-96a2-c70e5bfba511 admin admin] PATH_INFO: `/v3/projects/0043c22bf66f4ca1835b029e7e18b7d3/users/5da868013a794a0da1a16318c81a17b8/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:37.134369 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3edc11-ef42-430f-9307-98a333114826 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:37.136014 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3edc11-ef42-430f-9307-98a333114826 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:37.136465 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:37.148875 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 938/1928] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:37 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:37.155235 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfda58f-99fc-4442-96a2-c70e5bfba511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0043c22bf66f4ca1835b029e7e18b7d3, user_id=5da868013a794a0da1a16318c81a17b8, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:37.159014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfda58f-99fc-4442-96a2-c70e5bfba511 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:37.183148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 991/1929] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:37 2022] PUT /identity/v3/projects/0043c22bf66f4ca1835b029e7e18b7d3/users/5da868013a794a0da1a16318c81a17b8/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:37.195112 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a3e81-caed-4fdf-a9c4-b4e5dbed6284 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:37.195577 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a3e81-caed-4fdf-a9c4-b4e5dbed6284 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:37.195980 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a3e81-caed-4fdf-a9c4-b4e5dbed6284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:37.240386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-977a3e81-caed-4fdf-a9c4-b4e5dbed6284 None None] MFA Rules not processed for user `5da868013a794a0da1a16318c81a17b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:37.242569 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-977a3e81-caed-4fdf-a9c4-b4e5dbed6284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:37.415623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfda58f-99fc-4442-96a2-c70e5bfba511 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:37.417729 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4cfda58f-99fc-4442-96a2-c70e5bfba511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:37.425118 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 939/1930] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:37.426081 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a3e81-caed-4fdf-a9c4-b4e5dbed6284 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:37.497458 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfda58f-99fc-4442-96a2-c70e5bfba511 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:37.501627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e6196-d950-44c3-b207-d92efca11cb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:37.502328 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e6196-d950-44c3-b207-d92efca11cb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:37.502683 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e6196-d950-44c3-b207-d92efca11cb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:37.503122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e6196-d950-44c3-b207-d92efca11cb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:37.506014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f45e6196-d950-44c3-b207-d92efca11cb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:37.576536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a3e81-caed-4fdf-a9c4-b4e5dbed6284 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:37.582341 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee16ae0-425d-46a1-beb9-4d87c3da3e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:37.583762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee16ae0-425d-46a1-beb9-4d87c3da3e49 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:37.584264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee16ae0-425d-46a1-beb9-4d87c3da3e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:37.584893 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee16ae0-425d-46a1-beb9-4d87c3da3e49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:37.585812 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee16ae0-425d-46a1-beb9-4d87c3da3e49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:37.588402 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee16ae0-425d-46a1-beb9-4d87c3da3e49 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:37.598177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e6196-d950-44c3-b207-d92efca11cb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:37.600393 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e6196-d950-44c3-b207-d92efca11cb0 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:37.614008 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 992/1931] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:37.615925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1753f36-be80-4f7d-b359-5a30b3dcfe9a None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:37.616325 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1753f36-be80-4f7d-b359-5a30b3dcfe9a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:37.616671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1753f36-be80-4f7d-b359-5a30b3dcfe9a None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:37.618439 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 993/1932] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 29 20:33:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:33:37.624694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 940/1933] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:37 2022] POST /identity/v3/projects => generated 387 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:37.629876 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c257076-be51-476f-982a-e7698040f6e4 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:37.630217 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c257076-be51-476f-982a-e7698040f6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:37.630415 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee16ae0-425d-46a1-beb9-4d87c3da3e49 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:37.630576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c257076-be51-476f-982a-e7698040f6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:37.645571 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:37.665312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-5c257076-be51-476f-982a-e7698040f6e4 None None] MFA Rules not processed for user `c0c44754a1c04a2b87081aba6cb509d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:37.667715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5c257076-be51-476f-982a-e7698040f6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:37.752219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee16ae0-425d-46a1-beb9-4d87c3da3e49 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:37.784353 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3632c4-15b7-429f-b55a-04081c0675bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:37.785781 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3632c4-15b7-429f-b55a-04081c0675bf admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:37.786077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3632c4-15b7-429f-b55a-04081c0675bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:37.786352 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3632c4-15b7-429f-b55a-04081c0675bf admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:37.786997 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3632c4-15b7-429f-b55a-04081c0675bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:37.789222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3632c4-15b7-429f-b55a-04081c0675bf admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:37.809448 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 994/1934] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 29 20:33:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:37.831999 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c257076-be51-476f-982a-e7698040f6e4 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:37.835134 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5c257076-be51-476f-982a-e7698040f6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:37.836848 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 941/1935] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:37 2022] POST /identity/v3/users => generated 300 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:37.840905 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3632c4-15b7-429f-b55a-04081c0675bf admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:37.856766 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3632c4-15b7-429f-b55a-04081c0675bf admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:37.857134 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c257076-be51-476f-982a-e7698040f6e4 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:37.862188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30388a-20f6-45a1-bea5-e4c081a54466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:37.863126 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30388a-20f6-45a1-bea5-e4c081a54466 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:37.863516 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30388a-20f6-45a1-bea5-e4c081a54466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:37.863945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30388a-20f6-45a1-bea5-e4c081a54466 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:37.864836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30388a-20f6-45a1-bea5-e4c081a54466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:37.866687 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30388a-20f6-45a1-bea5-e4c081a54466 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:37.868683 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b244976-50b5-4d7e-8115-62808665231b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:37.869777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b244976-50b5-4d7e-8115-62808665231b service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:37.870271 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b244976-50b5-4d7e-8115-62808665231b service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:37.870857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b244976-50b5-4d7e-8115-62808665231b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:37.881333 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 942/1936] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:33:37.886178 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30388a-20f6-45a1-bea5-e4c081a54466 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:37.894220 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b244976-50b5-4d7e-8115-62808665231b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:37.898204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b244976-50b5-4d7e-8115-62808665231b service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:37.904524 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30388a-20f6-45a1-bea5-e4c081a54466 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:37.920420 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6522c-2b7a-4db6-9400-e326fba115c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:37.921268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 995/1937] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:33:37.922654 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6522c-2b7a-4db6-9400-e326fba115c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:37.922959 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6522c-2b7a-4db6-9400-e326fba115c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:37.923247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6522c-2b7a-4db6-9400-e326fba115c3 admin admin] PATH_INFO: `/v3/projects/9f7babd84ef942868c6319b65b05045d/users/97bf94057e244ba1a1f444d7e72eeeb6/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:37.945832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d6522c-2b7a-4db6-9400-e326fba115c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f7babd84ef942868c6319b65b05045d, user_id=97bf94057e244ba1a1f444d7e72eeeb6, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:37.953050 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d6522c-2b7a-4db6-9400-e326fba115c3 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:37.975189 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 943/1938] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:37 2022] PUT /identity/v3/projects/9f7babd84ef942868c6319b65b05045d/users/97bf94057e244ba1a1f444d7e72eeeb6/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:37.980844 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b244976-50b5-4d7e-8115-62808665231b service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:37.997037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b244976-50b5-4d7e-8115-62808665231b service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:38.016992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:38.106372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a89e4-912b-476c-b5bd-f1a7b9efc853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:38.107752 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a89e4-912b-476c-b5bd-f1a7b9efc853 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:38.107752 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a89e4-912b-476c-b5bd-f1a7b9efc853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:38.107752 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a89e4-912b-476c-b5bd-f1a7b9efc853 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:38.108404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1a89e4-912b-476c-b5bd-f1a7b9efc853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:38.109951 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1a89e4-912b-476c-b5bd-f1a7b9efc853 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:38.122578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 996/1939] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:38.126537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6522c-2b7a-4db6-9400-e326fba115c3 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:38.144135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6522c-2b7a-4db6-9400-e326fba115c3 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:38.149458 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786034c-566a-4681-ad67-c96d429d3333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:38.150224 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786034c-566a-4681-ad67-c96d429d3333 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:38.150561 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786034c-566a-4681-ad67-c96d429d3333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:38.150830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786034c-566a-4681-ad67-c96d429d3333 admin admin] PATH_INFO: `/v3/projects/9f7babd84ef942868c6319b65b05045d/users/97bf94057e244ba1a1f444d7e72eeeb6/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:38.152821 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786034c-566a-4681-ad67-c96d429d3333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f7babd84ef942868c6319b65b05045d, user_id=97bf94057e244ba1a1f444d7e72eeeb6, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:38.155589 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786034c-566a-4681-ad67-c96d429d3333 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:38.189748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 944/1940] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:38 2022] PUT /identity/v3/projects/9f7babd84ef942868c6319b65b05045d/users/97bf94057e244ba1a1f444d7e72eeeb6/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:38.191867 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e24f0-cf20-4200-94ee-60362d1e307f None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:38.192736 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e24f0-cf20-4200-94ee-60362d1e307f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:38.193084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e24f0-cf20-4200-94ee-60362d1e307f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:38.229333 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2b7e24f0-cf20-4200-94ee-60362d1e307f None None] MFA Rules not processed for user `97bf94057e244ba1a1f444d7e72eeeb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:38.230959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2b7e24f0-cf20-4200-94ee-60362d1e307f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:38.423820 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 997/1941] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:38.438732 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786034c-566a-4681-ad67-c96d429d3333 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:38.528347 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e24f0-cf20-4200-94ee-60362d1e307f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:38.542833 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786034c-566a-4681-ad67-c96d429d3333 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:38.551393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a8b07-65f6-45d0-bfec-6185e0750b8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:38.552380 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a8b07-65f6-45d0-bfec-6185e0750b8f service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:38.553033 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a8b07-65f6-45d0-bfec-6185e0750b8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:38.553526 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a8b07-65f6-45d0-bfec-6185e0750b8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:38.565514 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a8b07-65f6-45d0-bfec-6185e0750b8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:38.568522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519a8b07-65f6-45d0-bfec-6185e0750b8f service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:38.581145 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 945/1942] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:38.634155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e24f0-cf20-4200-94ee-60362d1e307f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:38.638691 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d175ea4-8bb4-414a-bf1c-b63c28ff4b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:38.639419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d175ea4-8bb4-414a-bf1c-b63c28ff4b47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:38.639769 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d175ea4-8bb4-414a-bf1c-b63c28ff4b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:38.640160 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d175ea4-8bb4-414a-bf1c-b63c28ff4b47 admin admin] PATH_INFO: `/v3/users/a171c3d8442a4000b977d8f484d25d3f` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:38.641377 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d175ea4-8bb4-414a-bf1c-b63c28ff4b47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a171c3d8442a4000b977d8f484d25d3f)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:38.651436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d175ea4-8bb4-414a-bf1c-b63c28ff4b47 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:38.690542 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a8b07-65f6-45d0-bfec-6185e0750b8f service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:38.702717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a8b07-65f6-45d0-bfec-6185e0750b8f service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:38.708787 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da0a09-10d3-431e-8ad3-a294d6655f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:38.709583 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da0a09-10d3-431e-8ad3-a294d6655f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:38.709913 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da0a09-10d3-431e-8ad3-a294d6655f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:38.710145 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da0a09-10d3-431e-8ad3-a294d6655f89 admin admin] PATH_INFO: `/v3/users/5da868013a794a0da1a16318c81a17b8` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:38.710502 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:38.711660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78da0a09-10d3-431e-8ad3-a294d6655f89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5da868013a794a0da1a16318c81a17b8)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:38.713486 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78da0a09-10d3-431e-8ad3-a294d6655f89 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:38.717903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6d175ea4-8bb4-414a-bf1c-b63c28ff4b47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a171c3d8442a4000b977d8f484d25d3f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:38.733417 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6d175ea4-8bb4-414a-bf1c-b63c28ff4b47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a171c3d8442a4000b977d8f484d25d3f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:38.742582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6d175ea4-8bb4-414a-bf1c-b63c28ff4b47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a171c3d8442a4000b977d8f484d25d3f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:38.743204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6d175ea4-8bb4-414a-bf1c-b63c28ff4b47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a171c3d8442a4000b977d8f484d25d3f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:38.743827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6d175ea4-8bb4-414a-bf1c-b63c28ff4b47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a171c3d8442a4000b977d8f484d25d3f'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:38.764919 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 998/1943] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:38 2022] DELETE /identity/v3/users/a171c3d8442a4000b977d8f484d25d3f => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:38.768654 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:38.769581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe5b9e-8348-4ab3-9d5c-cf819b9ae76f None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:38.770079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe5b9e-8348-4ab3-9d5c-cf819b9ae76f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:38.770489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe5b9e-8348-4ab3-9d5c-cf819b9ae76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:38.770622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-78da0a09-10d3-431e-8ad3-a294d6655f89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5da868013a794a0da1a16318c81a17b8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:38.781096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-78da0a09-10d3-431e-8ad3-a294d6655f89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5da868013a794a0da1a16318c81a17b8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:38.789267 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-78da0a09-10d3-431e-8ad3-a294d6655f89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5da868013a794a0da1a16318c81a17b8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:38.790011 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-78da0a09-10d3-431e-8ad3-a294d6655f89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5da868013a794a0da1a16318c81a17b8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:38.791743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4afe5b9e-8348-4ab3-9d5c-cf819b9ae76f None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:38.793054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4afe5b9e-8348-4ab3-9d5c-cf819b9ae76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:38.805461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-78da0a09-10d3-431e-8ad3-a294d6655f89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5da868013a794a0da1a16318c81a17b8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:38.811722 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 946/1944] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:38 2022] DELETE /identity/v3/users/5da868013a794a0da1a16318c81a17b8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:38.817805 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba79e99-cc58-496f-a26a-9f639454f2fc None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:38.818140 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba79e99-cc58-496f-a26a-9f639454f2fc None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:38.818418 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba79e99-cc58-496f-a26a-9f639454f2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:38.843170 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-cba79e99-cc58-496f-a26a-9f639454f2fc None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:38.844905 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-cba79e99-cc58-496f-a26a-9f639454f2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:38.866939 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 999/1945] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:38.877480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe5b9e-8348-4ab3-9d5c-cf819b9ae76f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:38.879035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4afe5b9e-8348-4ab3-9d5c-cf819b9ae76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:38.913610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 947/1946] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:38.924485 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba79e99-cc58-496f-a26a-9f639454f2fc None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:38.925955 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-cba79e99-cc58-496f-a26a-9f639454f2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:38.943011 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe5b9e-8348-4ab3-9d5c-cf819b9ae76f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:38.992493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba79e99-cc58-496f-a26a-9f639454f2fc None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:38.996308 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943aca38-2b9f-43a2-8f68-46a0b5863e72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:38.996924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943aca38-2b9f-43a2-8f68-46a0b5863e72 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:38.997205 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943aca38-2b9f-43a2-8f68-46a0b5863e72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:38.997479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943aca38-2b9f-43a2-8f68-46a0b5863e72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:39.006834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6a056-d837-486a-807e-690861dcb829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:39.007525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6a056-d837-486a-807e-690861dcb829 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:39.007777 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943aca38-2b9f-43a2-8f68-46a0b5863e72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:39.007870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6a056-d837-486a-807e-690861dcb829 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:39.008110 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6a056-d837-486a-807e-690861dcb829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:39.009353 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943aca38-2b9f-43a2-8f68-46a0b5863e72 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:39.019158 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6a056-d837-486a-807e-690861dcb829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:39.021213 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6a056-d837-486a-807e-690861dcb829 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:39.021686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 948/1947] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:33:39.032001 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1000/1948] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:33:39.431136 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943aca38-2b9f-43a2-8f68-46a0b5863e72 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:39.432956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-943aca38-2b9f-43a2-8f68-46a0b5863e72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:39.452053 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943aca38-2b9f-43a2-8f68-46a0b5863e72 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:39.466368 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6a056-d837-486a-807e-690861dcb829 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:39.467299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810fd30-83e0-4eed-8dae-b80659355d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:39.468050 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810fd30-83e0-4eed-8dae-b80659355d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:39.468380 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810fd30-83e0-4eed-8dae-b80659355d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:39.468648 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810fd30-83e0-4eed-8dae-b80659355d25 admin admin] PATH_INFO: `/v3/projects/e2f92e90a6fa46fb819389b138a9c0c1` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:39.470980 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-bfd6a056-d837-486a-807e-690861dcb829 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:39.481056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810fd30-83e0-4eed-8dae-b80659355d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2f92e90a6fa46fb819389b138a9c0c1)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:39.483274 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810fd30-83e0-4eed-8dae-b80659355d25 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:39.494422 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6a056-d837-486a-807e-690861dcb829 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:39.510004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e104e96-e495-42a3-8d96-48b102d4de8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:39.510787 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e104e96-e495-42a3-8d96-48b102d4de8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:39.511070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e104e96-e495-42a3-8d96-48b102d4de8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:39.511576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e104e96-e495-42a3-8d96-48b102d4de8a admin admin] PATH_INFO: `/v3/projects/0043c22bf66f4ca1835b029e7e18b7d3` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:39.513635 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e104e96-e495-42a3-8d96-48b102d4de8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0043c22bf66f4ca1835b029e7e18b7d3)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:39.519205 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e104e96-e495-42a3-8d96-48b102d4de8a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:39.543297 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:39.574153 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3810fd30-83e0-4eed-8dae-b80659355d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2f92e90a6fa46fb819389b138a9c0c1', 'request_id': 'req-3810fd30-83e0-4eed-8dae-b80659355d25'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:39.593965 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3810fd30-83e0-4eed-8dae-b80659355d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2f92e90a6fa46fb819389b138a9c0c1', 'request_id': 'req-3810fd30-83e0-4eed-8dae-b80659355d25'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:39.596792 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:39.603777 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3810fd30-83e0-4eed-8dae-b80659355d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2f92e90a6fa46fb819389b138a9c0c1', 'request_id': 'req-3810fd30-83e0-4eed-8dae-b80659355d25'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:39.610497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6e104e96-e495-42a3-8d96-48b102d4de8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0043c22bf66f4ca1835b029e7e18b7d3', 'request_id': 'req-6e104e96-e495-42a3-8d96-48b102d4de8a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:39.618923 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6e104e96-e495-42a3-8d96-48b102d4de8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0043c22bf66f4ca1835b029e7e18b7d3', 'request_id': 'req-6e104e96-e495-42a3-8d96-48b102d4de8a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:39.619915 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3810fd30-83e0-4eed-8dae-b80659355d25 admin admin] The token cache is being invalidate because project e2f92e90a6fa46fb819389b138a9c0c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:39.620415 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3810fd30-83e0-4eed-8dae-b80659355d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:39.621340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3810fd30-83e0-4eed-8dae-b80659355d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:39.623284 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 949/1949] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:39 2022] DELETE /identity/v3/projects/e2f92e90a6fa46fb819389b138a9c0c1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:39.626003 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6e104e96-e495-42a3-8d96-48b102d4de8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0043c22bf66f4ca1835b029e7e18b7d3', 'request_id': 'req-6e104e96-e495-42a3-8d96-48b102d4de8a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:39.627821 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810fd30-83e0-4eed-8dae-b80659355d25 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:39.630488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-3810fd30-83e0-4eed-8dae-b80659355d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:39.641103 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6e104e96-e495-42a3-8d96-48b102d4de8a admin admin] The token cache is being invalidate because project 0043c22bf66f4ca1835b029e7e18b7d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:39.641548 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6e104e96-e495-42a3-8d96-48b102d4de8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:39.642292 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6e104e96-e495-42a3-8d96-48b102d4de8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:39.644072 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1001/1950] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:39 2022] DELETE /identity/v3/projects/0043c22bf66f4ca1835b029e7e18b7d3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:39.648736 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e104e96-e495-42a3-8d96-48b102d4de8a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:39.650293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6e104e96-e495-42a3-8d96-48b102d4de8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:39.710966 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810fd30-83e0-4eed-8dae-b80659355d25 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:39.730654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e104e96-e495-42a3-8d96-48b102d4de8a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:39.737887 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3125007-bdbb-4897-8e4c-ea332032438e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:39.740194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3125007-bdbb-4897-8e4c-ea332032438e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:39.741021 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3125007-bdbb-4897-8e4c-ea332032438e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:39.741856 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3125007-bdbb-4897-8e4c-ea332032438e admin admin] PATH_INFO: `/v3/users/97bf94057e244ba1a1f444d7e72eeeb6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:39.744040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3125007-bdbb-4897-8e4c-ea332032438e admin admin] RBAC: Authorizing `identity:delete_user(user_id=97bf94057e244ba1a1f444d7e72eeeb6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:39.746663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3125007-bdbb-4897-8e4c-ea332032438e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:39.810213 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a7343-5017-4b9d-a30a-c43a2349dc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:39.812138 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a7343-5017-4b9d-a30a-c43a2349dc96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:39.812699 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a7343-5017-4b9d-a30a-c43a2349dc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:39.813120 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a7343-5017-4b9d-a30a-c43a2349dc96 admin admin] PATH_INFO: `/v3/users/78aed86bc2234c6daf0b8ce40d686e8d` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:39.814724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a7343-5017-4b9d-a30a-c43a2349dc96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78aed86bc2234c6daf0b8ce40d686e8d)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:39.815766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c3125007-bdbb-4897-8e4c-ea332032438e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97bf94057e244ba1a1f444d7e72eeeb6'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:39.817711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a7343-5017-4b9d-a30a-c43a2349dc96 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:39.837118 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c3125007-bdbb-4897-8e4c-ea332032438e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97bf94057e244ba1a1f444d7e72eeeb6'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:39.849815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c3125007-bdbb-4897-8e4c-ea332032438e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97bf94057e244ba1a1f444d7e72eeeb6'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:39.850717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c3125007-bdbb-4897-8e4c-ea332032438e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97bf94057e244ba1a1f444d7e72eeeb6'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:39.851358 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c3125007-bdbb-4897-8e4c-ea332032438e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97bf94057e244ba1a1f444d7e72eeeb6'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:39.879982 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1002/1951] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:39 2022] DELETE /identity/v3/users/97bf94057e244ba1a1f444d7e72eeeb6 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:39.881669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fe0a7343-5017-4b9d-a30a-c43a2349dc96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78aed86bc2234c6daf0b8ce40d686e8d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:39.891155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fe0a7343-5017-4b9d-a30a-c43a2349dc96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78aed86bc2234c6daf0b8ce40d686e8d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:39.901187 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fe0a7343-5017-4b9d-a30a-c43a2349dc96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78aed86bc2234c6daf0b8ce40d686e8d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:39.902084 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fe0a7343-5017-4b9d-a30a-c43a2349dc96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78aed86bc2234c6daf0b8ce40d686e8d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:39.924790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fe0a7343-5017-4b9d-a30a-c43a2349dc96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78aed86bc2234c6daf0b8ce40d686e8d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:39.944606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 950/1952] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:39 2022] DELETE /identity/v3/users/78aed86bc2234c6daf0b8ce40d686e8d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:40.242347 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3125007-bdbb-4897-8e4c-ea332032438e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:40.243963 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c3125007-bdbb-4897-8e4c-ea332032438e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:40.277324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:40.352619 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3125007-bdbb-4897-8e4c-ea332032438e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:40.354784 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a7343-5017-4b9d-a30a-c43a2349dc96 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:40.356985 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-fe0a7343-5017-4b9d-a30a-c43a2349dc96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:40.383210 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:40.419926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d5a66-6cb1-4ee5-87a6-d16c9f13e4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:40.420603 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d5a66-6cb1-4ee5-87a6-d16c9f13e4f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:40.420882 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d5a66-6cb1-4ee5-87a6-d16c9f13e4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:40.421189 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d5a66-6cb1-4ee5-87a6-d16c9f13e4f3 admin admin] PATH_INFO: `/v3/projects/c534a25c0fca42719e3c316eabd0def8` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:40.422617 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d5a66-6cb1-4ee5-87a6-d16c9f13e4f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c534a25c0fca42719e3c316eabd0def8)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:40.424547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d5a66-6cb1-4ee5-87a6-d16c9f13e4f3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:40.435922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a7343-5017-4b9d-a30a-c43a2349dc96 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:40.440939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81746d9-7861-435e-a378-0ed531af6e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:40.441576 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81746d9-7861-435e-a378-0ed531af6e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:40.441849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81746d9-7861-435e-a378-0ed531af6e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:40.442984 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81746d9-7861-435e-a378-0ed531af6e59 admin admin] PATH_INFO: `/v3/projects/9f7babd84ef942868c6319b65b05045d` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:40.444176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81746d9-7861-435e-a378-0ed531af6e59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f7babd84ef942868c6319b65b05045d)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:40.446143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81746d9-7861-435e-a378-0ed531af6e59 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:40.474055 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a27d5a66-6cb1-4ee5-87a6-d16c9f13e4f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c534a25c0fca42719e3c316eabd0def8', 'request_id': 'req-a27d5a66-6cb1-4ee5-87a6-d16c9f13e4f3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:40.481676 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a27d5a66-6cb1-4ee5-87a6-d16c9f13e4f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c534a25c0fca42719e3c316eabd0def8', 'request_id': 'req-a27d5a66-6cb1-4ee5-87a6-d16c9f13e4f3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:40.487860 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a27d5a66-6cb1-4ee5-87a6-d16c9f13e4f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c534a25c0fca42719e3c316eabd0def8', 'request_id': 'req-a27d5a66-6cb1-4ee5-87a6-d16c9f13e4f3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:40.498607 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d81746d9-7861-435e-a378-0ed531af6e59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f7babd84ef942868c6319b65b05045d', 'request_id': 'req-d81746d9-7861-435e-a378-0ed531af6e59'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:40.500254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a27d5a66-6cb1-4ee5-87a6-d16c9f13e4f3 admin admin] The token cache is being invalidate because project c534a25c0fca42719e3c316eabd0def8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:40.500550 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a27d5a66-6cb1-4ee5-87a6-d16c9f13e4f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:40.501439 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a27d5a66-6cb1-4ee5-87a6-d16c9f13e4f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:40.503010 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1003/1953] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:40 2022] DELETE /identity/v3/projects/c534a25c0fca42719e3c316eabd0def8 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:40.506497 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d81746d9-7861-435e-a378-0ed531af6e59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f7babd84ef942868c6319b65b05045d', 'request_id': 'req-d81746d9-7861-435e-a378-0ed531af6e59'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:40.508847 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cabb9-ae94-4669-acaa-06facef32858 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:40.509272 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cabb9-ae94-4669-acaa-06facef32858 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:40.509566 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cabb9-ae94-4669-acaa-06facef32858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:40.515297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d81746d9-7861-435e-a378-0ed531af6e59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f7babd84ef942868c6319b65b05045d', 'request_id': 'req-d81746d9-7861-435e-a378-0ed531af6e59'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:40.530026 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d81746d9-7861-435e-a378-0ed531af6e59 admin admin] The token cache is being invalidate because project 9f7babd84ef942868c6319b65b05045d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:40.530364 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d81746d9-7861-435e-a378-0ed531af6e59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:40.531152 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d81746d9-7861-435e-a378-0ed531af6e59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:40.532524 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-812cabb9-ae94-4669-acaa-06facef32858 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:40.532967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 951/1954] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:40 2022] DELETE /identity/v3/projects/9f7babd84ef942868c6319b65b05045d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:40.533960 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-812cabb9-ae94-4669-acaa-06facef32858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:40.538446 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c05b69-fe80-4933-96f5-e3efb33c631c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:40.538834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c05b69-fe80-4933-96f5-e3efb33c631c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:40.539269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c05b69-fe80-4933-96f5-e3efb33c631c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:40.562263 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-13c05b69-fe80-4933-96f5-e3efb33c631c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:40.563645 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-13c05b69-fe80-4933-96f5-e3efb33c631c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:40.598582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:40.620089 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1004/1955] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:40.621779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe21a2-8607-4740-9b23-b88a942804cb None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:40.622076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe21a2-8607-4740-9b23-b88a942804cb None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:40.622379 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe21a2-8607-4740-9b23-b88a942804cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:40.645873 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-fffe21a2-8607-4740-9b23-b88a942804cb None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:40.647286 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-fffe21a2-8607-4740-9b23-b88a942804cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:40.655909 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1005/1956] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:40.657579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 952/1957] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:40.666430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9d311-5a80-4a95-a450-6653c272bcc4 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:40.666716 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9d311-5a80-4a95-a450-6653c272bcc4 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:40.667004 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9d311-5a80-4a95-a450-6653c272bcc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:40.672406 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe21a2-8607-4740-9b23-b88a942804cb None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:40.691325 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe21a2-8607-4740-9b23-b88a942804cb None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:40.696384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee62b0b-2f3e-44eb-af27-ad21d24897c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:40.697020 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee62b0b-2f3e-44eb-af27-ad21d24897c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:40.697306 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee62b0b-2f3e-44eb-af27-ad21d24897c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:40.697585 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee62b0b-2f3e-44eb-af27-ad21d24897c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:40.698308 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee62b0b-2f3e-44eb-af27-ad21d24897c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:33:40.698621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee62b0b-2f3e-44eb-af27-ad21d24897c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:40.700368 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee62b0b-2f3e-44eb-af27-ad21d24897c3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:40.720162 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1006/1958] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:33:40.720162 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee62b0b-2f3e-44eb-af27-ad21d24897c3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:40.725545 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-79b9d311-5a80-4a95-a450-6653c272bcc4 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:40.727845 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-79b9d311-5a80-4a95-a450-6653c272bcc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:40.736436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee62b0b-2f3e-44eb-af27-ad21d24897c3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:40.742565 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4a859-165a-445b-b982-1cf9a29db170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:40.743326 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4a859-165a-445b-b982-1cf9a29db170 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:40.743748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4a859-165a-445b-b982-1cf9a29db170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:40.744197 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4a859-165a-445b-b982-1cf9a29db170 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:40.744739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 953/1959] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:40.744974 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4a859-165a-445b-b982-1cf9a29db170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:40.745605 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9d311-5a80-4a95-a450-6653c272bcc4 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:40.747228 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4a859-165a-445b-b982-1cf9a29db170 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:40.755758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9d311-5a80-4a95-a450-6653c272bcc4 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:40.761449 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96627e5-d95b-4d75-bab8-de567d54fad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:40.762467 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96627e5-d95b-4d75-bab8-de567d54fad9 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:40.762747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96627e5-d95b-4d75-bab8-de567d54fad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:40.763019 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96627e5-d95b-4d75-bab8-de567d54fad9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:40.763719 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96627e5-d95b-4d75-bab8-de567d54fad9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:33:40.764051 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96627e5-d95b-4d75-bab8-de567d54fad9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:40.765758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96627e5-d95b-4d75-bab8-de567d54fad9 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:40.780286 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 954/1960] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:33:40.780510 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1007/1961] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:40 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:40.784577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4a859-165a-445b-b982-1cf9a29db170 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:40.784743 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96627e5-d95b-4d75-bab8-de567d54fad9 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:40.795665 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96627e5-d95b-4d75-bab8-de567d54fad9 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:40.828918 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:40.897830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4a859-165a-445b-b982-1cf9a29db170 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:40.905192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f508d-65be-42f0-892b-2c733c054ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:40.905832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f508d-65be-42f0-892b-2c733c054ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:40.907674 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f508d-65be-42f0-892b-2c733c054ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:40.908189 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f508d-65be-42f0-892b-2c733c054ed3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:40.916696 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56377204-a974-4092-bc56-db4f0d644aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:40.917319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56377204-a974-4092-bc56-db4f0d644aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:40.917605 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56377204-a974-4092-bc56-db4f0d644aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:40.917863 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56377204-a974-4092-bc56-db4f0d644aa5 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:40.918569 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56377204-a974-4092-bc56-db4f0d644aa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:40.920499 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f508d-65be-42f0-892b-2c733c054ed3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:40.921671 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56377204-a974-4092-bc56-db4f0d644aa5 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:40.929982 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f508d-65be-42f0-892b-2c733c054ed3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:40.952013 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 955/1962] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:40 2022] POST /identity/v3/users => generated 287 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:40.953138 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56377204-a974-4092-bc56-db4f0d644aa5 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:40.968448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56377204-a974-4092-bc56-db4f0d644aa5 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:40.977450 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c09e6-5edc-428b-a7d9-74603c0f28d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:40.978069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c09e6-5edc-428b-a7d9-74603c0f28d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:40.978539 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c09e6-5edc-428b-a7d9-74603c0f28d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:40.978819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c09e6-5edc-428b-a7d9-74603c0f28d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:40.979462 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c09e6-5edc-428b-a7d9-74603c0f28d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:40.981175 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c09e6-5edc-428b-a7d9-74603c0f28d2 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:40.997112 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 956/1963] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:41.002617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c09e6-5edc-428b-a7d9-74603c0f28d2 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:41.014981 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c09e6-5edc-428b-a7d9-74603c0f28d2 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:41.017904 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1008/1964] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:40 2022] POST /identity/v3/projects => generated 385 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:41.019092 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f508d-65be-42f0-892b-2c733c054ed3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:41.123990 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a1273-987d-4261-9e25-9571c23a0ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:41.123990 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a1273-987d-4261-9e25-9571c23a0ace admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:41.123990 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a1273-987d-4261-9e25-9571c23a0ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:41.123990 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a1273-987d-4261-9e25-9571c23a0ace admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:41.124408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a1273-987d-4261-9e25-9571c23a0ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:41.124408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705a1273-987d-4261-9e25-9571c23a0ace admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:41.140106 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 957/1965] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:41.144247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a1273-987d-4261-9e25-9571c23a0ace admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:41.163899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a1273-987d-4261-9e25-9571c23a0ace admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:41.177903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f508d-65be-42f0-892b-2c733c054ed3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:41.181178 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a5189-7709-4d8e-b021-d0dc1ce037d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:41.182009 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a5189-7709-4d8e-b021-d0dc1ce037d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:41.182440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a5189-7709-4d8e-b021-d0dc1ce037d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:41.182852 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a5189-7709-4d8e-b021-d0dc1ce037d9 admin admin] PATH_INFO: `/v3/projects/f5b0e82510ba41beadc96db2928b8b29/users/3a0e7e23309942228b4a9570fa8332a1/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:41.184480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fb7ff-bb77-4067-97fd-6fefae3e17a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:41.185273 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fb7ff-bb77-4067-97fd-6fefae3e17a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:41.185718 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fb7ff-bb77-4067-97fd-6fefae3e17a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:41.185953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fb7ff-bb77-4067-97fd-6fefae3e17a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:41.186593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fb7ff-bb77-4067-97fd-6fefae3e17a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:41.190588 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fb7ff-bb77-4067-97fd-6fefae3e17a6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:41.216113 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a5189-7709-4d8e-b021-d0dc1ce037d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5b0e82510ba41beadc96db2928b8b29, user_id=3a0e7e23309942228b4a9570fa8332a1, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:41.220300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a5189-7709-4d8e-b021-d0dc1ce037d9 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:41.232964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1009/1966] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:41 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:41.237342 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fb7ff-bb77-4067-97fd-6fefae3e17a6 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:41.242203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 958/1967] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:41 2022] PUT /identity/v3/projects/f5b0e82510ba41beadc96db2928b8b29/users/3a0e7e23309942228b4a9570fa8332a1/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:41.252098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fb7ff-bb77-4067-97fd-6fefae3e17a6 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:41.258454 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba2a3d-d25b-4e11-b689-9840d268fac6 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:41.259159 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba2a3d-d25b-4e11-b689-9840d268fac6 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:41.259738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba2a3d-d25b-4e11-b689-9840d268fac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:41.298463 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-02ba2a3d-d25b-4e11-b689-9840d268fac6 None None] MFA Rules not processed for user `3a0e7e23309942228b4a9570fa8332a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:41.300471 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-02ba2a3d-d25b-4e11-b689-9840d268fac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:41.329091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:41.388845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e044ef-86e5-49de-8448-13e6c5f06beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:41.389810 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e044ef-86e5-49de-8448-13e6c5f06beb admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:41.390112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e044ef-86e5-49de-8448-13e6c5f06beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:41.390388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e044ef-86e5-49de-8448-13e6c5f06beb admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:41.391338 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e044ef-86e5-49de-8448-13e6c5f06beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:41.393852 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e044ef-86e5-49de-8448-13e6c5f06beb admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:41.410700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1010/1969] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:41.410890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 959/1969] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:41.414714 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e044ef-86e5-49de-8448-13e6c5f06beb admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:41.419879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba2a3d-d25b-4e11-b689-9840d268fac6 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:41.423484 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-02ba2a3d-d25b-4e11-b689-9840d268fac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:41.436267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e044ef-86e5-49de-8448-13e6c5f06beb admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:41.436419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:41.446681 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1309e84-f8f3-4e0c-844e-a55baed77b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:41.447334 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1309e84-f8f3-4e0c-844e-a55baed77b78 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:41.447663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1309e84-f8f3-4e0c-844e-a55baed77b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:41.448005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1309e84-f8f3-4e0c-844e-a55baed77b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:41.448709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1309e84-f8f3-4e0c-844e-a55baed77b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:41.450443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1309e84-f8f3-4e0c-844e-a55baed77b78 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:41.470143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1011/1970] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:41.471429 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1309e84-f8f3-4e0c-844e-a55baed77b78 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:41.482398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1309e84-f8f3-4e0c-844e-a55baed77b78 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:41.491235 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82217c1e-fe0f-41f5-8e3b-2abeecc02ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:41.492217 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82217c1e-fe0f-41f5-8e3b-2abeecc02ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:41.492635 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82217c1e-fe0f-41f5-8e3b-2abeecc02ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:41.493052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82217c1e-fe0f-41f5-8e3b-2abeecc02ccb admin admin] PATH_INFO: `/v3/projects/d898a6b1c9f94a17bbb98af4fe4e7c3e/users/fe0a7965d47449f8ab7683832e2ecfee/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:41.515089 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba2a3d-d25b-4e11-b689-9840d268fac6 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:41.515306 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82217c1e-fe0f-41f5-8e3b-2abeecc02ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d898a6b1c9f94a17bbb98af4fe4e7c3e, user_id=fe0a7965d47449f8ab7683832e2ecfee, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:41.518111 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82217c1e-fe0f-41f5-8e3b-2abeecc02ccb admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:41.519082 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e8697-4702-4902-b99b-2d93e719335a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:41.519738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e8697-4702-4902-b99b-2d93e719335a service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:41.520082 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e8697-4702-4902-b99b-2d93e719335a service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:41.520357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e8697-4702-4902-b99b-2d93e719335a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:41.537043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e8697-4702-4902-b99b-2d93e719335a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:41.538833 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e8697-4702-4902-b99b-2d93e719335a service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:41.543006 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1012/1971] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:41 2022] PUT /identity/v3/projects/d898a6b1c9f94a17bbb98af4fe4e7c3e/users/fe0a7965d47449f8ab7683832e2ecfee/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:41.554142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 960/1972] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:41.554424 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f43abe-f4e3-4498-aaf1-1c8acdd143d4 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:41.554543 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f43abe-f4e3-4498-aaf1-1c8acdd143d4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:41.554906 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f43abe-f4e3-4498-aaf1-1c8acdd143d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:41.580279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e8697-4702-4902-b99b-2d93e719335a service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:41.588442 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d5f43abe-f4e3-4498-aaf1-1c8acdd143d4 None None] MFA Rules not processed for user `fe0a7965d47449f8ab7683832e2ecfee`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:41.590238 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d5f43abe-f4e3-4498-aaf1-1c8acdd143d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:41.593815 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e8697-4702-4902-b99b-2d93e719335a service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:41.664199 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d2618-ef1f-4d05-b187-7b5b4115f36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:41.664921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d2618-ef1f-4d05-b187-7b5b4115f36f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:41.665261 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d2618-ef1f-4d05-b187-7b5b4115f36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:41.665537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d2618-ef1f-4d05-b187-7b5b4115f36f admin admin] PATH_INFO: `/v3/users/3a0e7e23309942228b4a9570fa8332a1` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:41.666914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d2618-ef1f-4d05-b187-7b5b4115f36f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a0e7e23309942228b4a9570fa8332a1)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:41.668998 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d2618-ef1f-4d05-b187-7b5b4115f36f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:41.670465 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1013/1973] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:41.675159 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f43abe-f4e3-4498-aaf1-1c8acdd143d4 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:41.692958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f43abe-f4e3-4498-aaf1-1c8acdd143d4 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:41.698281 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ae6da-3030-4350-b4af-a74673c2fc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:41.698933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ae6da-3030-4350-b4af-a74673c2fc8f admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:41.699225 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ae6da-3030-4350-b4af-a74673c2fc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:41.699484 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ae6da-3030-4350-b4af-a74673c2fc8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:41.700399 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ae6da-3030-4350-b4af-a74673c2fc8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:41.702588 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ae6da-3030-4350-b4af-a74673c2fc8f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:41.745429 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9c3d2618-ef1f-4d05-b187-7b5b4115f36f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a0e7e23309942228b4a9570fa8332a1'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:41.749401 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1014/1974] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:41 2022] POST /identity/v3/projects => generated 383 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:41.755679 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ae6da-3030-4350-b4af-a74673c2fc8f admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:41.762271 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9c3d2618-ef1f-4d05-b187-7b5b4115f36f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a0e7e23309942228b4a9570fa8332a1'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:41.776305 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9c3d2618-ef1f-4d05-b187-7b5b4115f36f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a0e7e23309942228b4a9570fa8332a1'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:41.777021 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9c3d2618-ef1f-4d05-b187-7b5b4115f36f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a0e7e23309942228b4a9570fa8332a1'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:41.798607 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9c3d2618-ef1f-4d05-b187-7b5b4115f36f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a0e7e23309942228b4a9570fa8332a1'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:41.816028 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 961/1975] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:41 2022] DELETE /identity/v3/users/3a0e7e23309942228b4a9570fa8332a1 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:41.823951 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f9133-6dad-43ef-9a8b-ca0fd4c5b904 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:41.824758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f9133-6dad-43ef-9a8b-ca0fd4c5b904 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:41.825415 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f9133-6dad-43ef-9a8b-ca0fd4c5b904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:41.855264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-be7f9133-6dad-43ef-9a8b-ca0fd4c5b904 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:41.856740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-be7f9133-6dad-43ef-9a8b-ca0fd4c5b904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:41.865374 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ae6da-3030-4350-b4af-a74673c2fc8f admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:41.871595 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:41.906101 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:41.964260 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285dab1d-04b4-4f88-afbe-2eb2a8140995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:41.966592 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285dab1d-04b4-4f88-afbe-2eb2a8140995 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:41.966878 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285dab1d-04b4-4f88-afbe-2eb2a8140995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:41.967206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285dab1d-04b4-4f88-afbe-2eb2a8140995 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:41.967844 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285dab1d-04b4-4f88-afbe-2eb2a8140995 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:41.970010 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285dab1d-04b4-4f88-afbe-2eb2a8140995 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:41.990284 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 962/1976] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:41.999363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1015/1977] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:41 2022] POST /identity/v3/users => generated 298 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:42.000684 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f9133-6dad-43ef-9a8b-ca0fd4c5b904 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:42.001126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285dab1d-04b4-4f88-afbe-2eb2a8140995 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:42.002252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-be7f9133-6dad-43ef-9a8b-ca0fd4c5b904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:42.003106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-285dab1d-04b4-4f88-afbe-2eb2a8140995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:42.017171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285dab1d-04b4-4f88-afbe-2eb2a8140995 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:42.022777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b8f4e-a55a-449c-b3be-edf1c7ffb643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:42.023366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b8f4e-a55a-449c-b3be-edf1c7ffb643 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:42.023653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b8f4e-a55a-449c-b3be-edf1c7ffb643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:42.024003 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b8f4e-a55a-449c-b3be-edf1c7ffb643 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:42.024666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b8f4e-a55a-449c-b3be-edf1c7ffb643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:42.026644 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b8f4e-a55a-449c-b3be-edf1c7ffb643 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:42.042449 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1016/1978] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:42.046595 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b8f4e-a55a-449c-b3be-edf1c7ffb643 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:42.058233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b8f4e-a55a-449c-b3be-edf1c7ffb643 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:42.063276 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0fbd7-65bb-40db-b656-cd1bdfad2cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:42.064167 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0fbd7-65bb-40db-b656-cd1bdfad2cc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:42.064519 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0fbd7-65bb-40db-b656-cd1bdfad2cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:42.064805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0fbd7-65bb-40db-b656-cd1bdfad2cc7 admin admin] PATH_INFO: `/v3/projects/b6e47b12cfc3402181784fc901d859d5/users/5a2330db9585492e9520143978bc3702/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:42.068511 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f9133-6dad-43ef-9a8b-ca0fd4c5b904 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:42.075662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2a8a6-3e85-4de2-ada9-3918d4ae1e60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:42.076457 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2a8a6-3e85-4de2-ada9-3918d4ae1e60 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:42.076832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2a8a6-3e85-4de2-ada9-3918d4ae1e60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:42.077015 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2a8a6-3e85-4de2-ada9-3918d4ae1e60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:42.082328 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0fbd7-65bb-40db-b656-cd1bdfad2cc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6e47b12cfc3402181784fc901d859d5, user_id=5a2330db9585492e9520143978bc3702, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:42.085119 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0fbd7-65bb-40db-b656-cd1bdfad2cc7 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:42.096833 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2a8a6-3e85-4de2-ada9-3918d4ae1e60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:42.099546 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2a8a6-3e85-4de2-ada9-3918d4ae1e60 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:42.100704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1017/1979] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:42 2022] PUT /identity/v3/projects/b6e47b12cfc3402181784fc901d859d5/users/5a2330db9585492e9520143978bc3702/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:42.104553 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0fbd7-65bb-40db-b656-cd1bdfad2cc7 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:42.114950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 963/1980] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:42.196308 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0fbd7-65bb-40db-b656-cd1bdfad2cc7 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:42.204423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa8e7b-6a69-4f97-8803-073eca41daa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:42.205380 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa8e7b-6a69-4f97-8803-073eca41daa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:42.205621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa8e7b-6a69-4f97-8803-073eca41daa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:42.205903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa8e7b-6a69-4f97-8803-073eca41daa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:42.206873 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa8e7b-6a69-4f97-8803-073eca41daa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:42.208951 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fa8e7b-6a69-4f97-8803-073eca41daa9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:42.222563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1018/1981] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:33:42.227514 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2a8a6-3e85-4de2-ada9-3918d4ae1e60 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:42.237970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2a8a6-3e85-4de2-ada9-3918d4ae1e60 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:42.244133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a073b01-9afb-438b-8167-2d41343fff2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:42.244723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a073b01-9afb-438b-8167-2d41343fff2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:42.245038 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a073b01-9afb-438b-8167-2d41343fff2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:42.245282 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a073b01-9afb-438b-8167-2d41343fff2d admin admin] PATH_INFO: `/v3/projects/b6e47b12cfc3402181784fc901d859d5/users/5a2330db9585492e9520143978bc3702/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:42.248301 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a073b01-9afb-438b-8167-2d41343fff2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6e47b12cfc3402181784fc901d859d5, user_id=5a2330db9585492e9520143978bc3702, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:42.252957 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a073b01-9afb-438b-8167-2d41343fff2d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:42.266441 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 964/1982] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:42 2022] PUT /identity/v3/projects/b6e47b12cfc3402181784fc901d859d5/users/5a2330db9585492e9520143978bc3702/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:42.278939 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88f6b9-036b-4d76-9a7a-ad51e423a963 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:42.278939 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88f6b9-036b-4d76-9a7a-ad51e423a963 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:42.278939 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88f6b9-036b-4d76-9a7a-ad51e423a963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:42.326412 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-3b88f6b9-036b-4d76-9a7a-ad51e423a963 None None] MFA Rules not processed for user `5a2330db9585492e9520143978bc3702`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:42.326412 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3b88f6b9-036b-4d76-9a7a-ad51e423a963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:42.412621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:42.447352 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1019/1983] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:42.462643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a073b01-9afb-438b-8167-2d41343fff2d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:42.465050 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2a073b01-9afb-438b-8167-2d41343fff2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:42.549493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a073b01-9afb-438b-8167-2d41343fff2d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:42.563454 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c85886d-179b-43a2-8240-f54de4894ac3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:42.570695 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85886d-179b-43a2-8240-f54de4894ac3 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:42.570695 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85886d-179b-43a2-8240-f54de4894ac3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:42.570695 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c85886d-179b-43a2-8240-f54de4894ac3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:42.570695 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2c85886d-179b-43a2-8240-f54de4894ac3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:42.594136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88f6b9-036b-4d76-9a7a-ad51e423a963 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:42.598065 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3b88f6b9-036b-4d76-9a7a-ad51e423a963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:42.666434 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85886d-179b-43a2-8240-f54de4894ac3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:42.669203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c85886d-179b-43a2-8240-f54de4894ac3 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:42.684455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 965/1984] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:42.706298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88f6b9-036b-4d76-9a7a-ad51e423a963 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:42.712996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0023d9-bf59-40bb-9675-6c8d818d5675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:42.713875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0023d9-bf59-40bb-9675-6c8d818d5675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:42.714264 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0023d9-bf59-40bb-9675-6c8d818d5675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:42.714558 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0023d9-bf59-40bb-9675-6c8d818d5675 admin admin] PATH_INFO: `/v3/projects/f5b0e82510ba41beadc96db2928b8b29` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:42.716000 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0023d9-bf59-40bb-9675-6c8d818d5675 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5b0e82510ba41beadc96db2928b8b29)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:42.718870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0023d9-bf59-40bb-9675-6c8d818d5675 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:42.789393 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5f0023d9-bf59-40bb-9675-6c8d818d5675 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5b0e82510ba41beadc96db2928b8b29', 'request_id': 'req-5f0023d9-bf59-40bb-9675-6c8d818d5675'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:42.798359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5f0023d9-bf59-40bb-9675-6c8d818d5675 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5b0e82510ba41beadc96db2928b8b29', 'request_id': 'req-5f0023d9-bf59-40bb-9675-6c8d818d5675'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:42.806044 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5f0023d9-bf59-40bb-9675-6c8d818d5675 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5b0e82510ba41beadc96db2928b8b29', 'request_id': 'req-5f0023d9-bf59-40bb-9675-6c8d818d5675'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:42.822187 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5f0023d9-bf59-40bb-9675-6c8d818d5675 admin admin] The token cache is being invalidate because project f5b0e82510ba41beadc96db2928b8b29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:42.822511 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5f0023d9-bf59-40bb-9675-6c8d818d5675 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:42.823576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5f0023d9-bf59-40bb-9675-6c8d818d5675 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:42.825521 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1020/1985] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:42 2022] DELETE /identity/v3/projects/f5b0e82510ba41beadc96db2928b8b29 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:42.838149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bae06-d2b0-4453-a7c9-6583520f5db2 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:42.838468 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bae06-d2b0-4453-a7c9-6583520f5db2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:42.838715 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bae06-d2b0-4453-a7c9-6583520f5db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:42.871558 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d35bae06-d2b0-4453-a7c9-6583520f5db2 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:42.872948 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d35bae06-d2b0-4453-a7c9-6583520f5db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:42.897400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:42.988908 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 966/1986] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:42.998127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c72bf-2330-459f-9ef4-820af561d454 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:42.998425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c72bf-2330-459f-9ef4-820af561d454 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:42.998695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c72bf-2330-459f-9ef4-820af561d454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:43.029925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-165c72bf-2330-459f-9ef4-820af561d454 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:43.031308 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-165c72bf-2330-459f-9ef4-820af561d454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:43.041375 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1021/1987] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:43.046350 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bae06-d2b0-4453-a7c9-6583520f5db2 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:43.063614 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bae06-d2b0-4453-a7c9-6583520f5db2 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:43.071468 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f16a6-464f-4903-b0fd-fa10e34d1ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:43.072360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f16a6-464f-4903-b0fd-fa10e34d1ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:43.072598 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f16a6-464f-4903-b0fd-fa10e34d1ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:43.072855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f16a6-464f-4903-b0fd-fa10e34d1ca6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:43.073926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f16a6-464f-4903-b0fd-fa10e34d1ca6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:33:43.073926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f16a6-464f-4903-b0fd-fa10e34d1ca6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:43.075656 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f16a6-464f-4903-b0fd-fa10e34d1ca6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:43.114048 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 967/1988] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:33:43.114941 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c72bf-2330-459f-9ef4-820af561d454 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:43.124827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c72bf-2330-459f-9ef4-820af561d454 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:43.133475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1c358-17e6-4be4-9a7e-5f6fded53010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:43.134612 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1c358-17e6-4be4-9a7e-5f6fded53010 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:43.135185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1c358-17e6-4be4-9a7e-5f6fded53010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:43.135719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1c358-17e6-4be4-9a7e-5f6fded53010 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:43.137393 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1c358-17e6-4be4-9a7e-5f6fded53010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:43.142356 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1c358-17e6-4be4-9a7e-5f6fded53010 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:43.197327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f16a6-464f-4903-b0fd-fa10e34d1ca6 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:43.203271 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1022/1989] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:43 2022] POST /identity/v3/projects => generated 359 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:43.209214 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f16a6-464f-4903-b0fd-fa10e34d1ca6 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:43.284905 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8275b19-224f-4c67-930c-9b2117763f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:43.285846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8275b19-224f-4c67-930c-9b2117763f8f admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:43.286177 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8275b19-224f-4c67-930c-9b2117763f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:43.287029 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8275b19-224f-4c67-930c-9b2117763f8f admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:43.287694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8275b19-224f-4c67-930c-9b2117763f8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:43.289815 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8275b19-224f-4c67-930c-9b2117763f8f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:43.317450 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 968/1990] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:43 2022] POST /identity/v3/users => generated 280 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:43.321917 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1c358-17e6-4be4-9a7e-5f6fded53010 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:43.334107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1c358-17e6-4be4-9a7e-5f6fded53010 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:43.338792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdca9a1-c732-40de-b14d-c6c085c7a1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:43.339403 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdca9a1-c732-40de-b14d-c6c085c7a1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:43.339681 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdca9a1-c732-40de-b14d-c6c085c7a1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:43.340000 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdca9a1-c732-40de-b14d-c6c085c7a1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:43.340639 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdca9a1-c732-40de-b14d-c6c085c7a1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:43.342218 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdca9a1-c732-40de-b14d-c6c085c7a1b0 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:43.356751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1023/1991] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:43.364670 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8275b19-224f-4c67-930c-9b2117763f8f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:43.376394 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8275b19-224f-4c67-930c-9b2117763f8f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:43.381899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637063f6-ee1b-45a6-bf30-cdf3d79330d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:43.382597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637063f6-ee1b-45a6-bf30-cdf3d79330d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:43.382882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637063f6-ee1b-45a6-bf30-cdf3d79330d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:43.383158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637063f6-ee1b-45a6-bf30-cdf3d79330d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:43.383813 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637063f6-ee1b-45a6-bf30-cdf3d79330d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:43.385482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637063f6-ee1b-45a6-bf30-cdf3d79330d6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:43.403564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 969/1992] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:33:43.407862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdca9a1-c732-40de-b14d-c6c085c7a1b0 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:43.419118 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdca9a1-c732-40de-b14d-c6c085c7a1b0 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:43.427513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64f2b6-6743-4bb8-ad61-7464f7b58e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:43.428748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64f2b6-6743-4bb8-ad61-7464f7b58e07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:43.429148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64f2b6-6743-4bb8-ad61-7464f7b58e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:43.429540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64f2b6-6743-4bb8-ad61-7464f7b58e07 admin admin] PATH_INFO: `/v3/projects/f291e03a2c134b62890e16ece69ef640/users/2f53f5ac4df24855b99d62b88792adb8/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:43.460938 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64f2b6-6743-4bb8-ad61-7464f7b58e07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f291e03a2c134b62890e16ece69ef640, user_id=2f53f5ac4df24855b99d62b88792adb8, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:43.463501 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64f2b6-6743-4bb8-ad61-7464f7b58e07 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:43.479978 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1024/1993] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:43 2022] PUT /identity/v3/projects/f291e03a2c134b62890e16ece69ef640/users/2f53f5ac4df24855b99d62b88792adb8/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:43.499113 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363bf57-2c9f-47a4-bff8-7e8e9cfd9e70 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:43.499524 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363bf57-2c9f-47a4-bff8-7e8e9cfd9e70 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:43.499890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363bf57-2c9f-47a4-bff8-7e8e9cfd9e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:43.548946 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-7363bf57-2c9f-47a4-bff8-7e8e9cfd9e70 None None] MFA Rules not processed for user `2f53f5ac4df24855b99d62b88792adb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:43.550527 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7363bf57-2c9f-47a4-bff8-7e8e9cfd9e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:43.660038 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 970/1994] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:43 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:43.707315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64f2b6-6743-4bb8-ad61-7464f7b58e07 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:43.709639 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2a64f2b6-6743-4bb8-ad61-7464f7b58e07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:43.722512 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:43.788356 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64f2b6-6743-4bb8-ad61-7464f7b58e07 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:43.792413 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb48470-5632-4c05-a000-cee2a6000d62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:43.793002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb48470-5632-4c05-a000-cee2a6000d62 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:43.793288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb48470-5632-4c05-a000-cee2a6000d62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:43.793565 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb48470-5632-4c05-a000-cee2a6000d62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:43.804563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb48470-5632-4c05-a000-cee2a6000d62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:43.806085 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb48470-5632-4c05-a000-cee2a6000d62 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:43.817304 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1025/1995] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:43 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:43.834335 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363bf57-2c9f-47a4-bff8-7e8e9cfd9e70 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:43.845698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:43.911919 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363bf57-2c9f-47a4-bff8-7e8e9cfd9e70 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:43.917964 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4435a1-50a0-4d3e-802a-373a57a459c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:43.918206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4435a1-50a0-4d3e-802a-373a57a459c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:43.918431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4435a1-50a0-4d3e-802a-373a57a459c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:43.918744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4435a1-50a0-4d3e-802a-373a57a459c4 admin admin] PATH_INFO: `/v3/users/2f53f5ac4df24855b99d62b88792adb8` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:43.919961 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4435a1-50a0-4d3e-802a-373a57a459c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f53f5ac4df24855b99d62b88792adb8)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:43.921964 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4435a1-50a0-4d3e-802a-373a57a459c4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:44.012233 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5f4435a1-50a0-4d3e-802a-373a57a459c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f53f5ac4df24855b99d62b88792adb8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:44.022646 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5f4435a1-50a0-4d3e-802a-373a57a459c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f53f5ac4df24855b99d62b88792adb8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:44.044105 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5f4435a1-50a0-4d3e-802a-373a57a459c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f53f5ac4df24855b99d62b88792adb8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:44.044105 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5f4435a1-50a0-4d3e-802a-373a57a459c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f53f5ac4df24855b99d62b88792adb8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:44.061940 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5f4435a1-50a0-4d3e-802a-373a57a459c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f53f5ac4df24855b99d62b88792adb8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:44.069051 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 971/1996] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:43 2022] DELETE /identity/v3/users/2f53f5ac4df24855b99d62b88792adb8 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:44.075439 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788f940-6d49-4f9a-9350-3c35148f6911 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:44.075730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788f940-6d49-4f9a-9350-3c35148f6911 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:44.076039 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788f940-6d49-4f9a-9350-3c35148f6911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:44.104997 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2788f940-6d49-4f9a-9350-3c35148f6911 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:44.106367 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2788f940-6d49-4f9a-9350-3c35148f6911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:44.178151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1026/1997] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:44.189967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4435a1-50a0-4d3e-802a-373a57a459c4 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:44.191735 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5f4435a1-50a0-4d3e-802a-373a57a459c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:44.313419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4435a1-50a0-4d3e-802a-373a57a459c4 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:44.339033 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9809d69-8992-48e8-9b6a-8afe9d478973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:44.339329 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9809d69-8992-48e8-9b6a-8afe9d478973 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:44.339797 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9809d69-8992-48e8-9b6a-8afe9d478973 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:44.340538 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9809d69-8992-48e8-9b6a-8afe9d478973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:44.363074 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9809d69-8992-48e8-9b6a-8afe9d478973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:44.368695 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9809d69-8992-48e8-9b6a-8afe9d478973 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:44.404602 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 972/1998] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:44.948007 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788f940-6d49-4f9a-9350-3c35148f6911 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:44.949925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2788f940-6d49-4f9a-9350-3c35148f6911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:44.962780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788f940-6d49-4f9a-9350-3c35148f6911 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:44.968079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6fcda-4990-4b52-84da-3957886187c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:44.969053 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6fcda-4990-4b52-84da-3957886187c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:44.969472 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6fcda-4990-4b52-84da-3957886187c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:44.969985 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6fcda-4990-4b52-84da-3957886187c5 admin admin] PATH_INFO: `/v3/projects/f291e03a2c134b62890e16ece69ef640` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:44.971248 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6fcda-4990-4b52-84da-3957886187c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f291e03a2c134b62890e16ece69ef640)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:44.973210 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6fcda-4990-4b52-84da-3957886187c5 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:45.040486 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-62c6fcda-4990-4b52-84da-3957886187c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f291e03a2c134b62890e16ece69ef640', 'request_id': 'req-62c6fcda-4990-4b52-84da-3957886187c5'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:45.054013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-62c6fcda-4990-4b52-84da-3957886187c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f291e03a2c134b62890e16ece69ef640', 'request_id': 'req-62c6fcda-4990-4b52-84da-3957886187c5'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:45.063170 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-62c6fcda-4990-4b52-84da-3957886187c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f291e03a2c134b62890e16ece69ef640', 'request_id': 'req-62c6fcda-4990-4b52-84da-3957886187c5'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:45.077483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-62c6fcda-4990-4b52-84da-3957886187c5 admin admin] The token cache is being invalidate because project f291e03a2c134b62890e16ece69ef640 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:45.077945 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-62c6fcda-4990-4b52-84da-3957886187c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:45.078722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-62c6fcda-4990-4b52-84da-3957886187c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:45.080710 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1027/1999] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:44 2022] DELETE /identity/v3/projects/f291e03a2c134b62890e16ece69ef640 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:45.087717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56de7dc-c100-4488-85d3-616b24eb1ad3 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:45.088282 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56de7dc-c100-4488-85d3-616b24eb1ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:45.088462 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56de7dc-c100-4488-85d3-616b24eb1ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:45.121863 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b56de7dc-c100-4488-85d3-616b24eb1ad3 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:45.123091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b56de7dc-c100-4488-85d3-616b24eb1ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:45.170768 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:45.232305 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 973/2000] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:45.241098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9a6fb-a802-48fa-ae29-fffd2d760576 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:45.241483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9a6fb-a802-48fa-ae29-fffd2d760576 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:45.241765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9a6fb-a802-48fa-ae29-fffd2d760576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:45.274594 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-75d9a6fb-a802-48fa-ae29-fffd2d760576 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:45.275919 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-75d9a6fb-a802-48fa-ae29-fffd2d760576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:45.284352 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1028/2001] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:45.289370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56de7dc-c100-4488-85d3-616b24eb1ad3 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:45.310505 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56de7dc-c100-4488-85d3-616b24eb1ad3 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:45.320598 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2129d5-f945-4b25-9452-c8efa01e2ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:45.321471 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2129d5-f945-4b25-9452-c8efa01e2ade admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:45.321762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2129d5-f945-4b25-9452-c8efa01e2ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:45.322050 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2129d5-f945-4b25-9452-c8efa01e2ade admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:45.322830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2129d5-f945-4b25-9452-c8efa01e2ade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:33:45.323142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2129d5-f945-4b25-9452-c8efa01e2ade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:45.324819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2129d5-f945-4b25-9452-c8efa01e2ade admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:45.346562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 974/2002] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:33:45.347903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9a6fb-a802-48fa-ae29-fffd2d760576 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:45.363212 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9a6fb-a802-48fa-ae29-fffd2d760576 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:45.369291 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa63c4-778a-4a5f-94ff-b62a1268b78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:45.369900 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa63c4-778a-4a5f-94ff-b62a1268b78a admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:45.370178 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa63c4-778a-4a5f-94ff-b62a1268b78a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:45.370452 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa63c4-778a-4a5f-94ff-b62a1268b78a admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:45.371100 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa63c4-778a-4a5f-94ff-b62a1268b78a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:45.373114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa63c4-778a-4a5f-94ff-b62a1268b78a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:45.416763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1029/2003] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:45 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:45.418061 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2129d5-f945-4b25-9452-c8efa01e2ade admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:45.429456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2129d5-f945-4b25-9452-c8efa01e2ade admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:45.525197 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d2c88-da1b-42bb-9666-f327cda1d910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:45.525986 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d2c88-da1b-42bb-9666-f327cda1d910 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:45.526411 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d2c88-da1b-42bb-9666-f327cda1d910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:45.527587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d2c88-da1b-42bb-9666-f327cda1d910 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:45.528567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d2c88-da1b-42bb-9666-f327cda1d910 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:45.530436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d2c88-da1b-42bb-9666-f327cda1d910 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:45.560209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 975/2004] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:45 2022] POST /identity/v3/users => generated 292 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:45.561668 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa63c4-778a-4a5f-94ff-b62a1268b78a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:45.576366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa63c4-778a-4a5f-94ff-b62a1268b78a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:45.581069 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a782f-480e-47b8-944d-00914c0a767c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:45.581687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a782f-480e-47b8-944d-00914c0a767c admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:45.581969 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a782f-480e-47b8-944d-00914c0a767c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:45.582260 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a782f-480e-47b8-944d-00914c0a767c admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:45.582894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a782f-480e-47b8-944d-00914c0a767c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:45.584648 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a782f-480e-47b8-944d-00914c0a767c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:45.586166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:45.597974 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1030/2005] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:45.602256 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d2c88-da1b-42bb-9666-f327cda1d910 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:45.613624 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d2c88-da1b-42bb-9666-f327cda1d910 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:45.619145 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa71e7-931f-459b-924f-06b0213d56a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:45.619762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa71e7-931f-459b-924f-06b0213d56a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:45.620070 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa71e7-931f-459b-924f-06b0213d56a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:45.620366 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa71e7-931f-459b-924f-06b0213d56a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:45.621019 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa71e7-931f-459b-924f-06b0213d56a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:45.622598 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa71e7-931f-459b-924f-06b0213d56a1 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:45.634832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 976/2006] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:45.639375 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a782f-480e-47b8-944d-00914c0a767c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:45.649959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a782f-480e-47b8-944d-00914c0a767c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:45.655406 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ef5ed-fe59-4c0e-befb-aedbf8d6160a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:45.656192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ef5ed-fe59-4c0e-befb-aedbf8d6160a admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:45.656623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ef5ed-fe59-4c0e-befb-aedbf8d6160a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:45.656905 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ef5ed-fe59-4c0e-befb-aedbf8d6160a admin admin] PATH_INFO: `/v3/projects/1f78f3c41743411f8b8395ea6c1357f0/users/ad6dbba0a02d4495a149edae3a56c408/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:45.673016 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72ef5ed-fe59-4c0e-befb-aedbf8d6160a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f78f3c41743411f8b8395ea6c1357f0, user_id=ad6dbba0a02d4495a149edae3a56c408, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:45.675862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72ef5ed-fe59-4c0e-befb-aedbf8d6160a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:45.690866 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1031/2007] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:45 2022] PUT /identity/v3/projects/1f78f3c41743411f8b8395ea6c1357f0/users/ad6dbba0a02d4495a149edae3a56c408/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:45.702180 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3835f-9429-4e22-a26b-7717a281d70e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:45.702462 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3835f-9429-4e22-a26b-7717a281d70e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:45.702733 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3835f-9429-4e22-a26b-7717a281d70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:45.736436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5ac3835f-9429-4e22-a26b-7717a281d70e None None] MFA Rules not processed for user `ad6dbba0a02d4495a149edae3a56c408`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:45.737809 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5ac3835f-9429-4e22-a26b-7717a281d70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:45.815005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 977/2008] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:45.821523 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ef5ed-fe59-4c0e-befb-aedbf8d6160a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:45.889636 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ef5ed-fe59-4c0e-befb-aedbf8d6160a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:45.894286 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629aac08-5eff-4090-b47d-32c3c62e9abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:45.894905 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629aac08-5eff-4090-b47d-32c3c62e9abe admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:45.895179 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629aac08-5eff-4090-b47d-32c3c62e9abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:45.895448 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629aac08-5eff-4090-b47d-32c3c62e9abe admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:45.896142 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629aac08-5eff-4090-b47d-32c3c62e9abe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:45.898100 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629aac08-5eff-4090-b47d-32c3c62e9abe admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:45.926292 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1032/2009] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:45 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:45.931360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3835f-9429-4e22-a26b-7717a281d70e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:45.942544 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3835f-9429-4e22-a26b-7717a281d70e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:45.970369 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:46.018137 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4ea95-1cf4-4670-bc42-f17ac4c8cc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:46.018751 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4ea95-1cf4-4670-bc42-f17ac4c8cc91 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:46.019027 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4ea95-1cf4-4670-bc42-f17ac4c8cc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:46.019291 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4ea95-1cf4-4670-bc42-f17ac4c8cc91 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:46.019925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4ea95-1cf4-4670-bc42-f17ac4c8cc91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:46.021928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4ea95-1cf4-4670-bc42-f17ac4c8cc91 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:46.053968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 978/2010] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:45 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:46.055305 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629aac08-5eff-4090-b47d-32c3c62e9abe admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:46.067187 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629aac08-5eff-4090-b47d-32c3c62e9abe admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:46.072699 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60f0a6-df66-40a3-89e1-75dd9b2ad011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:46.073328 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60f0a6-df66-40a3-89e1-75dd9b2ad011 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:46.073605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60f0a6-df66-40a3-89e1-75dd9b2ad011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:46.073914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f60f0a6-df66-40a3-89e1-75dd9b2ad011 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:46.074523 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f60f0a6-df66-40a3-89e1-75dd9b2ad011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:46.076184 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f60f0a6-df66-40a3-89e1-75dd9b2ad011 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:46.095136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1033/2011] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:46.095350 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4ea95-1cf4-4670-bc42-f17ac4c8cc91 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:46.106071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4ea95-1cf4-4670-bc42-f17ac4c8cc91 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:46.111324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca0df8-e0cb-41ee-a780-4e194984f511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:46.112015 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca0df8-e0cb-41ee-a780-4e194984f511 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:46.112299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca0df8-e0cb-41ee-a780-4e194984f511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:46.112618 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca0df8-e0cb-41ee-a780-4e194984f511 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:46.113288 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca0df8-e0cb-41ee-a780-4e194984f511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:46.115668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca0df8-e0cb-41ee-a780-4e194984f511 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:46.127027 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 979/2012] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:33:46.131760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60f0a6-df66-40a3-89e1-75dd9b2ad011 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:46.142275 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f60f0a6-df66-40a3-89e1-75dd9b2ad011 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:46.147683 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e29db8-7de8-45d9-9f8f-74a58f661577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:46.148432 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e29db8-7de8-45d9-9f8f-74a58f661577 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:46.148855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e29db8-7de8-45d9-9f8f-74a58f661577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:46.149141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e29db8-7de8-45d9-9f8f-74a58f661577 admin admin] PATH_INFO: `/v3/projects/7ed97be0bb7943b7a3562cf2412f8b51/users/3b190895c0e24daba533151decdc14f7/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:46.164823 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e29db8-7de8-45d9-9f8f-74a58f661577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ed97be0bb7943b7a3562cf2412f8b51, user_id=3b190895c0e24daba533151decdc14f7, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:46.167519 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e29db8-7de8-45d9-9f8f-74a58f661577 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:46.181701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1034/2013] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:46 2022] PUT /identity/v3/projects/7ed97be0bb7943b7a3562cf2412f8b51/users/3b190895c0e24daba533151decdc14f7/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:46.192071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522fff9-2522-4c12-a8c4-d68044b1707d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:46.192381 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522fff9-2522-4c12-a8c4-d68044b1707d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:46.193178 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522fff9-2522-4c12-a8c4-d68044b1707d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:46.225184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-4522fff9-2522-4c12-a8c4-d68044b1707d None None] MFA Rules not processed for user `3b190895c0e24daba533151decdc14f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:46.226413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4522fff9-2522-4c12-a8c4-d68044b1707d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:46.300372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 980/2014] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:46.305041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e29db8-7de8-45d9-9f8f-74a58f661577 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:46.379858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e29db8-7de8-45d9-9f8f-74a58f661577 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:46.385653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce5f99-3cdf-44c0-a2ef-dcff3aa6bc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:46.386429 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce5f99-3cdf-44c0-a2ef-dcff3aa6bc6b admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:46.386723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce5f99-3cdf-44c0-a2ef-dcff3aa6bc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:46.387229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce5f99-3cdf-44c0-a2ef-dcff3aa6bc6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:46.388224 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ce5f99-3cdf-44c0-a2ef-dcff3aa6bc6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:46.390028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ce5f99-3cdf-44c0-a2ef-dcff3aa6bc6b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:46.429206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1035/2015] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:46 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:46.430409 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4522fff9-2522-4c12-a8c4-d68044b1707d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:46.450330 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4522fff9-2522-4c12-a8c4-d68044b1707d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:46.531145 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1db4b7-560f-46a4-b6f4-312974f3065c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:46.531786 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1db4b7-560f-46a4-b6f4-312974f3065c admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:46.532087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1db4b7-560f-46a4-b6f4-312974f3065c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:46.532355 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1db4b7-560f-46a4-b6f4-312974f3065c admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:46.533003 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1db4b7-560f-46a4-b6f4-312974f3065c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:46.535195 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1db4b7-560f-46a4-b6f4-312974f3065c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:46.558860 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 981/2016] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:46 2022] POST /identity/v3/users => generated 298 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:46.563387 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce5f99-3cdf-44c0-a2ef-dcff3aa6bc6b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:46.575810 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce5f99-3cdf-44c0-a2ef-dcff3aa6bc6b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:46.580578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb060f0c-dab6-49f2-9aea-41050e8e8c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:46.581186 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb060f0c-dab6-49f2-9aea-41050e8e8c80 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:46.581460 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb060f0c-dab6-49f2-9aea-41050e8e8c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:46.581729 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb060f0c-dab6-49f2-9aea-41050e8e8c80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:46.582406 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb060f0c-dab6-49f2-9aea-41050e8e8c80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:46.584551 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb060f0c-dab6-49f2-9aea-41050e8e8c80 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:46.585971 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:46.597853 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1036/2017] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:46.602781 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1db4b7-560f-46a4-b6f4-312974f3065c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:46.614946 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1db4b7-560f-46a4-b6f4-312974f3065c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:46.620181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298cfa98-f6fb-4145-b54b-ec0b7aed8dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:46.620934 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cfa98-f6fb-4145-b54b-ec0b7aed8dd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:46.621327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cfa98-f6fb-4145-b54b-ec0b7aed8dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:46.621488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cfa98-f6fb-4145-b54b-ec0b7aed8dd0 admin admin] PATH_INFO: `/v3/projects/60b7ec2b97404072a76a0570180f4acb/users/51b4793370d04c3d941c8c351e60c078/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:46.640510 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298cfa98-f6fb-4145-b54b-ec0b7aed8dd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b7ec2b97404072a76a0570180f4acb, user_id=51b4793370d04c3d941c8c351e60c078, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:46.643768 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298cfa98-f6fb-4145-b54b-ec0b7aed8dd0 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:46.656210 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:46.664026 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 982/2018] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:46 2022] PUT /identity/v3/projects/60b7ec2b97404072a76a0570180f4acb/users/51b4793370d04c3d941c8c351e60c078/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:46.670913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb060f0c-dab6-49f2-9aea-41050e8e8c80 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:46.681406 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb060f0c-dab6-49f2-9aea-41050e8e8c80 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:46.741589 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154e81e-babf-48d2-a387-701e1dc8dae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:46.742128 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154e81e-babf-48d2-a387-701e1dc8dae8 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:46.742409 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154e81e-babf-48d2-a387-701e1dc8dae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:46.742682 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154e81e-babf-48d2-a387-701e1dc8dae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:46.743328 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f154e81e-babf-48d2-a387-701e1dc8dae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:46.744944 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f154e81e-babf-48d2-a387-701e1dc8dae8 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:46.755996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1037/2019] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:46.760209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298cfa98-f6fb-4145-b54b-ec0b7aed8dd0 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:46.772919 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298cfa98-f6fb-4145-b54b-ec0b7aed8dd0 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:46.777537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a67bef-3141-405a-8406-692b5cb875b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:46.778199 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a67bef-3141-405a-8406-692b5cb875b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:46.778478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a67bef-3141-405a-8406-692b5cb875b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:46.778799 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a67bef-3141-405a-8406-692b5cb875b2 admin admin] PATH_INFO: `/v3/projects/60b7ec2b97404072a76a0570180f4acb/users/51b4793370d04c3d941c8c351e60c078/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:46.780765 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a67bef-3141-405a-8406-692b5cb875b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b7ec2b97404072a76a0570180f4acb, user_id=51b4793370d04c3d941c8c351e60c078, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:46.783418 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a67bef-3141-405a-8406-692b5cb875b2 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:46.794038 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 983/2020] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:46 2022] PUT /identity/v3/projects/60b7ec2b97404072a76a0570180f4acb/users/51b4793370d04c3d941c8c351e60c078/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:46.805736 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b50d9-f3bd-4484-bc0d-ee5d5a567f03 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:46.806086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b50d9-f3bd-4484-bc0d-ee5d5a567f03 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:46.806376 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b50d9-f3bd-4484-bc0d-ee5d5a567f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:46.839662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-045b50d9-f3bd-4484-bc0d-ee5d5a567f03 None None] MFA Rules not processed for user `51b4793370d04c3d941c8c351e60c078`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:46.840937 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-045b50d9-f3bd-4484-bc0d-ee5d5a567f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:46.930350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1038/2021] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:46 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:46.944470 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a67bef-3141-405a-8406-692b5cb875b2 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:46.946891 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-47a67bef-3141-405a-8406-692b5cb875b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:47.013897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a67bef-3141-405a-8406-692b5cb875b2 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:47.017378 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295edfe-1498-4a2c-85a9-27a6fd56b803 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:47.017984 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295edfe-1498-4a2c-85a9-27a6fd56b803 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:47.018263 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295edfe-1498-4a2c-85a9-27a6fd56b803 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:47.018537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295edfe-1498-4a2c-85a9-27a6fd56b803 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:47.096893 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295edfe-1498-4a2c-85a9-27a6fd56b803 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:47.098349 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295edfe-1498-4a2c-85a9-27a6fd56b803 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:47.109095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 984/2022] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:48.711246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b50d9-f3bd-4484-bc0d-ee5d5a567f03 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:48.712699 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-045b50d9-f3bd-4484-bc0d-ee5d5a567f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:48.836302 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b50d9-f3bd-4484-bc0d-ee5d5a567f03 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:48.841557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9ff13-5a55-432a-9a3c-4356dab89920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:48.842221 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9ff13-5a55-432a-9a3c-4356dab89920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:48.842510 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9ff13-5a55-432a-9a3c-4356dab89920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:48.842957 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9ff13-5a55-432a-9a3c-4356dab89920 admin admin] PATH_INFO: `/v3/users/1d38175008134363be61247e645ebe70` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:48.844792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a9ff13-5a55-432a-9a3c-4356dab89920 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d38175008134363be61247e645ebe70)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:48.846976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a9ff13-5a55-432a-9a3c-4356dab89920 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:48.879015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:48.944517 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-75a9ff13-5a55-432a-9a3c-4356dab89920 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d38175008134363be61247e645ebe70'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:48.961628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-75a9ff13-5a55-432a-9a3c-4356dab89920 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d38175008134363be61247e645ebe70'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:48.974845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-75a9ff13-5a55-432a-9a3c-4356dab89920 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d38175008134363be61247e645ebe70'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:48.975738 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-75a9ff13-5a55-432a-9a3c-4356dab89920 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d38175008134363be61247e645ebe70'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:48.977035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-75a9ff13-5a55-432a-9a3c-4356dab89920 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d38175008134363be61247e645ebe70'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:49.007079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1039/2023] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:48 2022] DELETE /identity/v3/users/1d38175008134363be61247e645ebe70 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:49.011969 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af3396-cd47-4749-b2a6-c7afcda94d79 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:49.012518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af3396-cd47-4749-b2a6-c7afcda94d79 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:49.012686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af3396-cd47-4749-b2a6-c7afcda94d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:49.035627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b0af3396-cd47-4749-b2a6-c7afcda94d79 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:49.039322 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b0af3396-cd47-4749-b2a6-c7afcda94d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:49.126505 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 985/2024] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:49.140608 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9ff13-5a55-432a-9a3c-4356dab89920 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:49.141999 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-75a9ff13-5a55-432a-9a3c-4356dab89920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:49.229704 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af3396-cd47-4749-b2a6-c7afcda94d79 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:49.231217 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b0af3396-cd47-4749-b2a6-c7afcda94d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:49.308753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9ff13-5a55-432a-9a3c-4356dab89920 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:49.324735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719deb2a-fba2-4f00-b483-b6f034089f81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:49.325743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719deb2a-fba2-4f00-b483-b6f034089f81 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:49.326026 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719deb2a-fba2-4f00-b483-b6f034089f81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:49.326303 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719deb2a-fba2-4f00-b483-b6f034089f81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:49.337603 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719deb2a-fba2-4f00-b483-b6f034089f81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:49.338714 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:49.339492 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719deb2a-fba2-4f00-b483-b6f034089f81 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:49.361636 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1040/2025] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:33:49.402820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af3396-cd47-4749-b2a6-c7afcda94d79 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:49.412338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f765788-0cd4-4861-9fff-c0983b71ee65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:49.413478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f765788-0cd4-4861-9fff-c0983b71ee65 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:49.414056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f765788-0cd4-4861-9fff-c0983b71ee65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:49.414785 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f765788-0cd4-4861-9fff-c0983b71ee65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:49.418494 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7f765788-0cd4-4861-9fff-c0983b71ee65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:49.553138 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f765788-0cd4-4861-9fff-c0983b71ee65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:49.554656 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f765788-0cd4-4861-9fff-c0983b71ee65 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:49.578140 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 986/2026] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:33:49.869309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719deb2a-fba2-4f00-b483-b6f034089f81 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:49.871670 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-719deb2a-fba2-4f00-b483-b6f034089f81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:49.899694 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719deb2a-fba2-4f00-b483-b6f034089f81 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:49.911140 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703d22b-665b-422b-afbc-840485d6a1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:49.912837 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703d22b-665b-422b-afbc-840485d6a1bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:49.913151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703d22b-665b-422b-afbc-840485d6a1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:49.913544 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703d22b-665b-422b-afbc-840485d6a1bb admin admin] PATH_INFO: `/v3/projects/b075f5d497314bef9a9ebf2fd5d5a1d2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:49.915012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703d22b-665b-422b-afbc-840485d6a1bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b075f5d497314bef9a9ebf2fd5d5a1d2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:49.918965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703d22b-665b-422b-afbc-840485d6a1bb admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:50.041896 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4703d22b-665b-422b-afbc-840485d6a1bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b075f5d497314bef9a9ebf2fd5d5a1d2', 'request_id': 'req-4703d22b-665b-422b-afbc-840485d6a1bb'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:50.061206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4703d22b-665b-422b-afbc-840485d6a1bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b075f5d497314bef9a9ebf2fd5d5a1d2', 'request_id': 'req-4703d22b-665b-422b-afbc-840485d6a1bb'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:50.075375 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4703d22b-665b-422b-afbc-840485d6a1bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b075f5d497314bef9a9ebf2fd5d5a1d2', 'request_id': 'req-4703d22b-665b-422b-afbc-840485d6a1bb'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:50.100821 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4703d22b-665b-422b-afbc-840485d6a1bb admin admin] The token cache is being invalidate because project b075f5d497314bef9a9ebf2fd5d5a1d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:50.101564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4703d22b-665b-422b-afbc-840485d6a1bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:50.103339 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4703d22b-665b-422b-afbc-840485d6a1bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:50.107658 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1041/2027] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:49 2022] DELETE /identity/v3/projects/b075f5d497314bef9a9ebf2fd5d5a1d2 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:50.111770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f765788-0cd4-4861-9fff-c0983b71ee65 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:50.125585 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f765788-0cd4-4861-9fff-c0983b71ee65 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:50.310679 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c01ba6-552b-4cf4-93a9-0f0e8f4b048d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:50.311602 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c01ba6-552b-4cf4-93a9-0f0e8f4b048d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:50.312121 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c01ba6-552b-4cf4-93a9-0f0e8f4b048d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:50.312606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c01ba6-552b-4cf4-93a9-0f0e8f4b048d admin admin] PATH_INFO: `/v3/users/7a2df26e6e9e418e923a5b3bfe344b73` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:50.314340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c01ba6-552b-4cf4-93a9-0f0e8f4b048d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a2df26e6e9e418e923a5b3bfe344b73)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:50.316641 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c01ba6-552b-4cf4-93a9-0f0e8f4b048d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:50.421637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f6c01ba6-552b-4cf4-93a9-0f0e8f4b048d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a2df26e6e9e418e923a5b3bfe344b73'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:50.445311 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f6c01ba6-552b-4cf4-93a9-0f0e8f4b048d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a2df26e6e9e418e923a5b3bfe344b73'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:50.461346 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f6c01ba6-552b-4cf4-93a9-0f0e8f4b048d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a2df26e6e9e418e923a5b3bfe344b73'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:50.463317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f6c01ba6-552b-4cf4-93a9-0f0e8f4b048d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a2df26e6e9e418e923a5b3bfe344b73'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:50.494729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f6c01ba6-552b-4cf4-93a9-0f0e8f4b048d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a2df26e6e9e418e923a5b3bfe344b73'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:50.510269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 987/2028] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:50 2022] DELETE /identity/v3/users/7a2df26e6e9e418e923a5b3bfe344b73 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:50.646946 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703d22b-665b-422b-afbc-840485d6a1bb admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:50.649428 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4703d22b-665b-422b-afbc-840485d6a1bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:50.725456 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:50.810064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703d22b-665b-422b-afbc-840485d6a1bb admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:50.916287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d7cfd-d170-4397-b3a5-041283527102 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:50.917780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d7cfd-d170-4397-b3a5-041283527102 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:50.918542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d7cfd-d170-4397-b3a5-041283527102 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:50.919039 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d7cfd-d170-4397-b3a5-041283527102 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:50.921802 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c16d7cfd-d170-4397-b3a5-041283527102 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:51.128423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d7cfd-d170-4397-b3a5-041283527102 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:51.128423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d7cfd-d170-4397-b3a5-041283527102 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:51.142870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1042/2029] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:51.310911 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c01ba6-552b-4cf4-93a9-0f0e8f4b048d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:51.313164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f6c01ba6-552b-4cf4-93a9-0f0e8f4b048d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:51.383853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:51.489748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c01ba6-552b-4cf4-93a9-0f0e8f4b048d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:51.495628 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced3a2c-7781-4b4c-89ff-1f5ecbd233fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:51.496332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced3a2c-7781-4b4c-89ff-1f5ecbd233fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:51.496618 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced3a2c-7781-4b4c-89ff-1f5ecbd233fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:51.496893 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced3a2c-7781-4b4c-89ff-1f5ecbd233fd admin admin] PATH_INFO: `/v3/projects/e2ad6bacd0a94d66ab4285164b912fa2` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:51.498066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced3a2c-7781-4b4c-89ff-1f5ecbd233fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2ad6bacd0a94d66ab4285164b912fa2)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:51.500689 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced3a2c-7781-4b4c-89ff-1f5ecbd233fd admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:51.582484 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2ced3a2c-7781-4b4c-89ff-1f5ecbd233fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2ad6bacd0a94d66ab4285164b912fa2', 'request_id': 'req-2ced3a2c-7781-4b4c-89ff-1f5ecbd233fd'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:51.591061 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2ced3a2c-7781-4b4c-89ff-1f5ecbd233fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2ad6bacd0a94d66ab4285164b912fa2', 'request_id': 'req-2ced3a2c-7781-4b4c-89ff-1f5ecbd233fd'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:51.602712 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2ced3a2c-7781-4b4c-89ff-1f5ecbd233fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2ad6bacd0a94d66ab4285164b912fa2', 'request_id': 'req-2ced3a2c-7781-4b4c-89ff-1f5ecbd233fd'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:51.650111 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2ced3a2c-7781-4b4c-89ff-1f5ecbd233fd admin admin] The token cache is being invalidate because project e2ad6bacd0a94d66ab4285164b912fa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:51.650240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2ced3a2c-7781-4b4c-89ff-1f5ecbd233fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:51.651627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2ced3a2c-7781-4b4c-89ff-1f5ecbd233fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:51.666128 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 988/2030] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:51 2022] DELETE /identity/v3/projects/e2ad6bacd0a94d66ab4285164b912fa2 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:51.671428 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5cb14-340a-4b77-9284-01be5cdead91 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:51.671780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5cb14-340a-4b77-9284-01be5cdead91 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:51.672127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5cb14-340a-4b77-9284-01be5cdead91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:51.719540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-6fd5cb14-340a-4b77-9284-01be5cdead91 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:51.721172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6fd5cb14-340a-4b77-9284-01be5cdead91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:51.856709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1043/2031] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:51.865738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86a684-d0c7-4825-888d-30b095ab89c3 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:51.866204 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86a684-d0c7-4825-888d-30b095ab89c3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:51.866548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86a684-d0c7-4825-888d-30b095ab89c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:51.897276 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-dc86a684-d0c7-4825-888d-30b095ab89c3 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:51.898721 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-dc86a684-d0c7-4825-888d-30b095ab89c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:51.909313 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 989/2032] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:51.914506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5cb14-340a-4b77-9284-01be5cdead91 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:51.925581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5cb14-340a-4b77-9284-01be5cdead91 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:51.930878 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101086a-ca82-4aa0-b6cb-4e9fe8e4e39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:51.931662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101086a-ca82-4aa0-b6cb-4e9fe8e4e39f admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:51.932147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101086a-ca82-4aa0-b6cb-4e9fe8e4e39f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:51.932816 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101086a-ca82-4aa0-b6cb-4e9fe8e4e39f admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:51.936754 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101086a-ca82-4aa0-b6cb-4e9fe8e4e39f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:33:51.937077 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101086a-ca82-4aa0-b6cb-4e9fe8e4e39f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:51.938710 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101086a-ca82-4aa0-b6cb-4e9fe8e4e39f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:51.957769 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1044/2033] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:33:51.963575 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86a684-d0c7-4825-888d-30b095ab89c3 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:51.978831 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86a684-d0c7-4825-888d-30b095ab89c3 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:51.984423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03b8bc-3193-4921-8611-286967f0b767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:51.985121 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03b8bc-3193-4921-8611-286967f0b767 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:51.985606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03b8bc-3193-4921-8611-286967f0b767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:51.986068 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03b8bc-3193-4921-8611-286967f0b767 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:51.987174 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03b8bc-3193-4921-8611-286967f0b767 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:51.989374 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03b8bc-3193-4921-8611-286967f0b767 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:52.054438 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 990/2034] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:51 2022] POST /identity/v3/projects => generated 365 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:52.059144 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101086a-ca82-4aa0-b6cb-4e9fe8e4e39f admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:52.072922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101086a-ca82-4aa0-b6cb-4e9fe8e4e39f admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:52.098727 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:52.165719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e06d84f-4f40-4515-b6da-8d837f22179a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:52.166990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06d84f-4f40-4515-b6da-8d837f22179a admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:52.167524 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06d84f-4f40-4515-b6da-8d837f22179a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:52.167898 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06d84f-4f40-4515-b6da-8d837f22179a admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:52.168908 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e06d84f-4f40-4515-b6da-8d837f22179a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:52.170951 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e06d84f-4f40-4515-b6da-8d837f22179a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:52.203372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1045/2035] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:52 2022] POST /identity/v3/users => generated 283 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:52.204890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03b8bc-3193-4921-8611-286967f0b767 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:52.220836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03b8bc-3193-4921-8611-286967f0b767 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:52.228676 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f5a77-efce-408e-a2b0-1d88ea8948f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:52.230146 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f5a77-efce-408e-a2b0-1d88ea8948f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:52.230657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f5a77-efce-408e-a2b0-1d88ea8948f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:52.231146 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f5a77-efce-408e-a2b0-1d88ea8948f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:52.232066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f5a77-efce-408e-a2b0-1d88ea8948f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:52.235738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f5a77-efce-408e-a2b0-1d88ea8948f3 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:52.251983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 991/2036] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:52.258553 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e06d84f-4f40-4515-b6da-8d837f22179a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:52.272388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e06d84f-4f40-4515-b6da-8d837f22179a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:52.280821 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6bef3e-7e73-47ff-bcab-9856cedcd90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:52.281655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bef3e-7e73-47ff-bcab-9856cedcd90a admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:52.282050 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bef3e-7e73-47ff-bcab-9856cedcd90a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:52.282464 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6bef3e-7e73-47ff-bcab-9856cedcd90a admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:52.283230 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6bef3e-7e73-47ff-bcab-9856cedcd90a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:52.291404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6bef3e-7e73-47ff-bcab-9856cedcd90a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:52.318959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1046/2037] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:52.323233 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f5a77-efce-408e-a2b0-1d88ea8948f3 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:52.335055 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f5a77-efce-408e-a2b0-1d88ea8948f3 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:52.341176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409be8c-95f3-4c7e-9784-e8bdaf31f16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:52.342577 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409be8c-95f3-4c7e-9784-e8bdaf31f16e admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:52.343015 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409be8c-95f3-4c7e-9784-e8bdaf31f16e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:52.343473 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409be8c-95f3-4c7e-9784-e8bdaf31f16e admin admin] PATH_INFO: `/v3/projects/70334af7f020436cafe97073b94c2a7e/users/ea4a04348d1d41039020d277b6c1c98c/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:52.367137 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409be8c-95f3-4c7e-9784-e8bdaf31f16e admin admin] RBAC: Authorizing `identity:create_grant(project_id=70334af7f020436cafe97073b94c2a7e, user_id=ea4a04348d1d41039020d277b6c1c98c, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:52.369990 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409be8c-95f3-4c7e-9784-e8bdaf31f16e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:52.416087 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e4b96-c614-482f-9511-a4eed79bc1ce None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:52.417672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e4b96-c614-482f-9511-a4eed79bc1ce None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:52.418056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e4b96-c614-482f-9511-a4eed79bc1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:52.422644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 992/2038] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:52 2022] PUT /identity/v3/projects/70334af7f020436cafe97073b94c2a7e/users/ea4a04348d1d41039020d277b6c1c98c/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:52.460781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-da9e4b96-c614-482f-9511-a4eed79bc1ce None None] MFA Rules not processed for user `ea4a04348d1d41039020d277b6c1c98c`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:52.462153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-da9e4b96-c614-482f-9511-a4eed79bc1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:52.506484 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409be8c-95f3-4c7e-9784-e8bdaf31f16e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:52.508657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d409be8c-95f3-4c7e-9784-e8bdaf31f16e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:52.565727 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1047/2039] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:52.570375 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e4b96-c614-482f-9511-a4eed79bc1ce None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:52.581891 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:52.612123 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409be8c-95f3-4c7e-9784-e8bdaf31f16e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:52.616525 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31e015-efa5-48ab-9d05-6100f85cc116 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:52.617663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31e015-efa5-48ab-9d05-6100f85cc116 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:52.618161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31e015-efa5-48ab-9d05-6100f85cc116 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:52.618598 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31e015-efa5-48ab-9d05-6100f85cc116 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:52.620592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-3a31e015-efa5-48ab-9d05-6100f85cc116 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:52.670379 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e4b96-c614-482f-9511-a4eed79bc1ce None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:52.675926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea44bb7-9d2d-4ca9-b7e9-9f68073ece98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:52.676949 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea44bb7-9d2d-4ca9-b7e9-9f68073ece98 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:52.677335 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea44bb7-9d2d-4ca9-b7e9-9f68073ece98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:52.677808 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea44bb7-9d2d-4ca9-b7e9-9f68073ece98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:52.678515 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea44bb7-9d2d-4ca9-b7e9-9f68073ece98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:52.680598 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea44bb7-9d2d-4ca9-b7e9-9f68073ece98 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:52.708680 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31e015-efa5-48ab-9d05-6100f85cc116 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:52.710754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31e015-efa5-48ab-9d05-6100f85cc116 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:52.723332 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1048/2040] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:52 2022] POST /identity/v3/projects => generated 367 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:52.726949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 993/2041] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:33:52.728406 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea44bb7-9d2d-4ca9-b7e9-9f68073ece98 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:52.749547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:52.862532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea44bb7-9d2d-4ca9-b7e9-9f68073ece98 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:52.883675 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259636c-5900-48fa-a020-a97f3edb678f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:52.884388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259636c-5900-48fa-a020-a97f3edb678f admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:52.886062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259636c-5900-48fa-a020-a97f3edb678f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:52.886350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259636c-5900-48fa-a020-a97f3edb678f admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:52.887016 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2259636c-5900-48fa-a020-a97f3edb678f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:52.890476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2259636c-5900-48fa-a020-a97f3edb678f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:52.925246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1049/2042] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:52 2022] POST /identity/v3/users => generated 290 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:52.926354 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31e015-efa5-48ab-9d05-6100f85cc116 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:52.942861 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31e015-efa5-48ab-9d05-6100f85cc116 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:52.950109 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0531d-6929-41a7-996d-63b32ea61f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:52.951262 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0531d-6929-41a7-996d-63b32ea61f81 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:52.951708 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0531d-6929-41a7-996d-63b32ea61f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:52.952307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0531d-6929-41a7-996d-63b32ea61f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:52.953302 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0531d-6929-41a7-996d-63b32ea61f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:52.956003 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0531d-6929-41a7-996d-63b32ea61f81 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:52.980583 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259636c-5900-48fa-a020-a97f3edb678f admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:52.981129 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 994/2043] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:52.992041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259636c-5900-48fa-a020-a97f3edb678f admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:53.010663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ff905-a139-4643-a5f2-b197ca414362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:53.011389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ff905-a139-4643-a5f2-b197ca414362 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:53.011673 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ff905-a139-4643-a5f2-b197ca414362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:53.013854 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ff905-a139-4643-a5f2-b197ca414362 admin admin] PATH_INFO: `/v3/projects/c106941afde44402a5bf2cf04c543eef/users/e7c0f550253244f38a0d1741791ecb88/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:53.033106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ff905-a139-4643-a5f2-b197ca414362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c106941afde44402a5bf2cf04c543eef, user_id=e7c0f550253244f38a0d1741791ecb88, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:53.037539 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ff905-a139-4643-a5f2-b197ca414362 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:53.055268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1050/2044] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:52 2022] PUT /identity/v3/projects/c106941afde44402a5bf2cf04c543eef/users/e7c0f550253244f38a0d1741791ecb88/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:53.059904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0531d-6929-41a7-996d-63b32ea61f81 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:53.077992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0531d-6929-41a7-996d-63b32ea61f81 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:53.210482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733b049-dba2-4f32-8c9e-b56096f5249a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:53.211208 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733b049-dba2-4f32-8c9e-b56096f5249a admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:53.211437 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733b049-dba2-4f32-8c9e-b56096f5249a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:53.211714 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733b049-dba2-4f32-8c9e-b56096f5249a admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:53.212396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733b049-dba2-4f32-8c9e-b56096f5249a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:53.214019 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733b049-dba2-4f32-8c9e-b56096f5249a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:53.227446 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 995/2045] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:53.233092 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ff905-a139-4643-a5f2-b197ca414362 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:53.258630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ff905-a139-4643-a5f2-b197ca414362 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:53.264624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e7d2b-e232-490e-b8f5-de7f690c96a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:53.265537 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e7d2b-e232-490e-b8f5-de7f690c96a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:53.265823 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e7d2b-e232-490e-b8f5-de7f690c96a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:53.266265 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e7d2b-e232-490e-b8f5-de7f690c96a9 admin admin] PATH_INFO: `/v3/projects/c106941afde44402a5bf2cf04c543eef/users/e7c0f550253244f38a0d1741791ecb88/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:53.275832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e7d2b-e232-490e-b8f5-de7f690c96a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c106941afde44402a5bf2cf04c543eef, user_id=e7c0f550253244f38a0d1741791ecb88, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:53.281620 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e7d2b-e232-490e-b8f5-de7f690c96a9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:53.308738 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1051/2046] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:53 2022] PUT /identity/v3/projects/c106941afde44402a5bf2cf04c543eef/users/e7c0f550253244f38a0d1741791ecb88/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:53.321580 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2e29b-f523-49c8-bece-d6e3f0b85472 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:53.322136 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2e29b-f523-49c8-bece-d6e3f0b85472 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:53.323109 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2e29b-f523-49c8-bece-d6e3f0b85472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:53.364806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-9db2e29b-f523-49c8-bece-d6e3f0b85472 None None] MFA Rules not processed for user `e7c0f550253244f38a0d1741791ecb88`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:53.366205 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9db2e29b-f523-49c8-bece-d6e3f0b85472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:53.458138 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 996/2047] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:53 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:53.485455 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e7d2b-e232-490e-b8f5-de7f690c96a9 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:53.487498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-123e7d2b-e232-490e-b8f5-de7f690c96a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:53.571399 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e7d2b-e232-490e-b8f5-de7f690c96a9 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:53.589310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22035e50-0191-4ea8-9342-cca9e8fac6df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:53.592432 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22035e50-0191-4ea8-9342-cca9e8fac6df service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:53.592760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22035e50-0191-4ea8-9342-cca9e8fac6df service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:53.593071 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22035e50-0191-4ea8-9342-cca9e8fac6df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:53.616956 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22035e50-0191-4ea8-9342-cca9e8fac6df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:53.618446 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22035e50-0191-4ea8-9342-cca9e8fac6df service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:53.633715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1052/2048] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:53 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:53.796703 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2e29b-f523-49c8-bece-d6e3f0b85472 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:53.803133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:53.811185 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2e29b-f523-49c8-bece-d6e3f0b85472 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:53.822218 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902200a8-684e-4910-87ca-1d8b50b522cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:53.822989 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902200a8-684e-4910-87ca-1d8b50b522cf service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:53.823532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902200a8-684e-4910-87ca-1d8b50b522cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:53.824071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902200a8-684e-4910-87ca-1d8b50b522cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:53.906673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902200a8-684e-4910-87ca-1d8b50b522cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:53.908911 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902200a8-684e-4910-87ca-1d8b50b522cf service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:53.923780 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 997/2049] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:54.022959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22035e50-0191-4ea8-9342-cca9e8fac6df service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:54.024581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-22035e50-0191-4ea8-9342-cca9e8fac6df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:54.136266 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22035e50-0191-4ea8-9342-cca9e8fac6df service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:54.148311 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ca29a-07ad-41ec-8f58-dcbb97f9c74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:54.151601 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ca29a-07ad-41ec-8f58-dcbb97f9c74e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:54.153192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ca29a-07ad-41ec-8f58-dcbb97f9c74e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:54.153497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ca29a-07ad-41ec-8f58-dcbb97f9c74e admin admin] PATH_INFO: `/v3/users/17f987e2b97849bca2e92bbce21f2674` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:54.157339 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ca29a-07ad-41ec-8f58-dcbb97f9c74e admin admin] RBAC: Authorizing `identity:delete_user(user_id=17f987e2b97849bca2e92bbce21f2674)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:54.161883 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ca29a-07ad-41ec-8f58-dcbb97f9c74e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:54.174779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:54.273321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-610ca29a-07ad-41ec-8f58-dcbb97f9c74e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17f987e2b97849bca2e92bbce21f2674'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:54.291277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-610ca29a-07ad-41ec-8f58-dcbb97f9c74e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17f987e2b97849bca2e92bbce21f2674'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:54.317977 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-610ca29a-07ad-41ec-8f58-dcbb97f9c74e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17f987e2b97849bca2e92bbce21f2674'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:54.321319 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-610ca29a-07ad-41ec-8f58-dcbb97f9c74e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17f987e2b97849bca2e92bbce21f2674'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:54.322065 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-610ca29a-07ad-41ec-8f58-dcbb97f9c74e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17f987e2b97849bca2e92bbce21f2674'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:54.375906 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1053/2050] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:54 2022] DELETE /identity/v3/users/17f987e2b97849bca2e92bbce21f2674 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:54.381842 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576ae1d-bc9f-4799-91b3-72b4397bbdbc None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:54.382621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576ae1d-bc9f-4799-91b3-72b4397bbdbc None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:54.383230 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576ae1d-bc9f-4799-91b3-72b4397bbdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:54.446754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-1576ae1d-bc9f-4799-91b3-72b4397bbdbc None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:54.448424 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1576ae1d-bc9f-4799-91b3-72b4397bbdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:54.572605 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 998/2051] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:54.589442 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ca29a-07ad-41ec-8f58-dcbb97f9c74e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:54.591522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-610ca29a-07ad-41ec-8f58-dcbb97f9c74e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:54.806516 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ca29a-07ad-41ec-8f58-dcbb97f9c74e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:54.820778 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4f1c5-176b-49cb-b427-19177533a35a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:54.821446 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4f1c5-176b-49cb-b427-19177533a35a service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:54.821715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4f1c5-176b-49cb-b427-19177533a35a service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:54.821999 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4f1c5-176b-49cb-b427-19177533a35a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:54.837042 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4f1c5-176b-49cb-b427-19177533a35a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:54.839019 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4f1c5-176b-49cb-b427-19177533a35a service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:54.854648 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1054/2052] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:55.406338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576ae1d-bc9f-4799-91b3-72b4397bbdbc None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:55.407887 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1576ae1d-bc9f-4799-91b3-72b4397bbdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:55.422293 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576ae1d-bc9f-4799-91b3-72b4397bbdbc None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:55.429843 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba27c0-91a5-4641-bb7e-e4e0e6fa7b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:55.430560 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba27c0-91a5-4641-bb7e-e4e0e6fa7b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:55.430838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba27c0-91a5-4641-bb7e-e4e0e6fa7b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:55.431290 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba27c0-91a5-4641-bb7e-e4e0e6fa7b79 admin admin] PATH_INFO: `/v3/projects/1268ac632b7d43289d77234d055d537c` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:55.432877 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba27c0-91a5-4641-bb7e-e4e0e6fa7b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1268ac632b7d43289d77234d055d537c)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:55.435280 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba27c0-91a5-4641-bb7e-e4e0e6fa7b79 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:55.512941 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fcba27c0-91a5-4641-bb7e-e4e0e6fa7b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1268ac632b7d43289d77234d055d537c', 'request_id': 'req-fcba27c0-91a5-4641-bb7e-e4e0e6fa7b79'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:55.522204 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fcba27c0-91a5-4641-bb7e-e4e0e6fa7b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1268ac632b7d43289d77234d055d537c', 'request_id': 'req-fcba27c0-91a5-4641-bb7e-e4e0e6fa7b79'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:55.530820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fcba27c0-91a5-4641-bb7e-e4e0e6fa7b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1268ac632b7d43289d77234d055d537c', 'request_id': 'req-fcba27c0-91a5-4641-bb7e-e4e0e6fa7b79'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:55.544231 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fcba27c0-91a5-4641-bb7e-e4e0e6fa7b79 admin admin] The token cache is being invalidate because project 1268ac632b7d43289d77234d055d537c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:55.544583 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fcba27c0-91a5-4641-bb7e-e4e0e6fa7b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:55.545348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fcba27c0-91a5-4641-bb7e-e4e0e6fa7b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:55.546998 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 999/2053] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:55 2022] DELETE /identity/v3/projects/1268ac632b7d43289d77234d055d537c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:55.551893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4f1c5-176b-49cb-b427-19177533a35a service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:55.564679 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4f1c5-176b-49cb-b427-19177533a35a service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:55.638131 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ee668-ed82-4fe2-97de-85540fade214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:55.639495 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ee668-ed82-4fe2-97de-85540fade214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:55.639962 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ee668-ed82-4fe2-97de-85540fade214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:55.640632 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ee668-ed82-4fe2-97de-85540fade214 admin admin] PATH_INFO: `/v3/users/aa0aefbcfb2740cfad0c0cd6b7e1a031` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:55.642509 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ee668-ed82-4fe2-97de-85540fade214 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa0aefbcfb2740cfad0c0cd6b7e1a031)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:55.645395 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ee668-ed82-4fe2-97de-85540fade214 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:55.703335 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:55.742745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8a4ee668-ed82-4fe2-97de-85540fade214 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa0aefbcfb2740cfad0c0cd6b7e1a031'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:55.754125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8a4ee668-ed82-4fe2-97de-85540fade214 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa0aefbcfb2740cfad0c0cd6b7e1a031'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:55.767847 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8a4ee668-ed82-4fe2-97de-85540fade214 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa0aefbcfb2740cfad0c0cd6b7e1a031'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:55.771084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8a4ee668-ed82-4fe2-97de-85540fade214 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa0aefbcfb2740cfad0c0cd6b7e1a031'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:55.773124 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8a4ee668-ed82-4fe2-97de-85540fade214 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa0aefbcfb2740cfad0c0cd6b7e1a031'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:55.810214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1055/2054] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:55 2022] DELETE /identity/v3/users/aa0aefbcfb2740cfad0c0cd6b7e1a031 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:56.180383 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba27c0-91a5-4641-bb7e-e4e0e6fa7b79 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:56.182992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-fcba27c0-91a5-4641-bb7e-e4e0e6fa7b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:56.220229 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba27c0-91a5-4641-bb7e-e4e0e6fa7b79 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:56.243902 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:56.313657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ef446-d0bf-40e7-b82a-b2d4b822a88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:56.314319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ef446-d0bf-40e7-b82a-b2d4b822a88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:56.314596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ef446-d0bf-40e7-b82a-b2d4b822a88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:56.314861 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ef446-d0bf-40e7-b82a-b2d4b822a88f admin admin] PATH_INFO: `/v3/projects/1612b88af26d41158cee3dbcd8f54e24` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:56.316029 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ef446-d0bf-40e7-b82a-b2d4b822a88f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1612b88af26d41158cee3dbcd8f54e24)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:56.317921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ef446-d0bf-40e7-b82a-b2d4b822a88f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:56.382535 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1b6ef446-d0bf-40e7-b82a-b2d4b822a88f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1612b88af26d41158cee3dbcd8f54e24', 'request_id': 'req-1b6ef446-d0bf-40e7-b82a-b2d4b822a88f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:56.393649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1b6ef446-d0bf-40e7-b82a-b2d4b822a88f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1612b88af26d41158cee3dbcd8f54e24', 'request_id': 'req-1b6ef446-d0bf-40e7-b82a-b2d4b822a88f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:56.409038 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1b6ef446-d0bf-40e7-b82a-b2d4b822a88f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1612b88af26d41158cee3dbcd8f54e24', 'request_id': 'req-1b6ef446-d0bf-40e7-b82a-b2d4b822a88f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:56.429344 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1b6ef446-d0bf-40e7-b82a-b2d4b822a88f admin admin] The token cache is being invalidate because project 1612b88af26d41158cee3dbcd8f54e24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:56.429630 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1b6ef446-d0bf-40e7-b82a-b2d4b822a88f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:56.430824 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-1b6ef446-d0bf-40e7-b82a-b2d4b822a88f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:56.433339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1000/2055] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:56 2022] DELETE /identity/v3/projects/1612b88af26d41158cee3dbcd8f54e24 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:56.441046 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbebe71-7f70-4cdd-a855-012f057efe50 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:56.441340 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbebe71-7f70-4cdd-a855-012f057efe50 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:56.441623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbebe71-7f70-4cdd-a855-012f057efe50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:56.466798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-6cbebe71-7f70-4cdd-a855-012f057efe50 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:56.468216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6cbebe71-7f70-4cdd-a855-012f057efe50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:56.542682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ef446-d0bf-40e7-b82a-b2d4b822a88f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:56.544155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1b6ef446-d0bf-40e7-b82a-b2d4b822a88f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:56.546965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1056/2056] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:56.555330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57611e9f-7639-4db8-ae2a-2d3d7c8cdb9d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:56.555977 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57611e9f-7639-4db8-ae2a-2d3d7c8cdb9d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:56.556071 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57611e9f-7639-4db8-ae2a-2d3d7c8cdb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:56.564779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ef446-d0bf-40e7-b82a-b2d4b822a88f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:56.572630 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0db1ad-4564-48d5-8e99-93a6a2d94048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:56.574115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0db1ad-4564-48d5-8e99-93a6a2d94048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:56.574496 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0db1ad-4564-48d5-8e99-93a6a2d94048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:56.574974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0db1ad-4564-48d5-8e99-93a6a2d94048 admin admin] PATH_INFO: `/v3/users/ad6dbba0a02d4495a149edae3a56c408` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:56.576876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0db1ad-4564-48d5-8e99-93a6a2d94048 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad6dbba0a02d4495a149edae3a56c408)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:56.579225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0db1ad-4564-48d5-8e99-93a6a2d94048 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:56.584722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-57611e9f-7639-4db8-ae2a-2d3d7c8cdb9d None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:56.585939 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-57611e9f-7639-4db8-ae2a-2d3d7c8cdb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:56.601504 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1057/2057] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:56.608237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57611e9f-7639-4db8-ae2a-2d3d7c8cdb9d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:56.622298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57611e9f-7639-4db8-ae2a-2d3d7c8cdb9d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:56.638807 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71450255-e771-44b3-873a-a62aa101a204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:56.639529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71450255-e771-44b3-873a-a62aa101a204 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:56.639999 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71450255-e771-44b3-873a-a62aa101a204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:56.640439 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71450255-e771-44b3-873a-a62aa101a204 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:56.641519 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71450255-e771-44b3-873a-a62aa101a204 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:33:56.641973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71450255-e771-44b3-873a-a62aa101a204 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:56.643950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71450255-e771-44b3-873a-a62aa101a204 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:56.650421 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fc0db1ad-4564-48d5-8e99-93a6a2d94048 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad6dbba0a02d4495a149edae3a56c408'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:56.659642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1058/2058] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:33:56.664570 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71450255-e771-44b3-873a-a62aa101a204 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:56.666041 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fc0db1ad-4564-48d5-8e99-93a6a2d94048 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad6dbba0a02d4495a149edae3a56c408'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:56.675231 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fc0db1ad-4564-48d5-8e99-93a6a2d94048 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad6dbba0a02d4495a149edae3a56c408'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:56.676496 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fc0db1ad-4564-48d5-8e99-93a6a2d94048 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad6dbba0a02d4495a149edae3a56c408'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:56.682983 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71450255-e771-44b3-873a-a62aa101a204 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:56.688314 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f407cc4-12c4-47ef-beeb-a6a8c743c991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:56.689014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f407cc4-12c4-47ef-beeb-a6a8c743c991 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:56.689469 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f407cc4-12c4-47ef-beeb-a6a8c743c991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:56.689618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f407cc4-12c4-47ef-beeb-a6a8c743c991 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:56.690334 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f407cc4-12c4-47ef-beeb-a6a8c743c991 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:56.692554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f407cc4-12c4-47ef-beeb-a6a8c743c991 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:56.704820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fc0db1ad-4564-48d5-8e99-93a6a2d94048 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad6dbba0a02d4495a149edae3a56c408'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:56.716246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1001/2059] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:56 2022] DELETE /identity/v3/users/ad6dbba0a02d4495a149edae3a56c408 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:56.717733 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438fde3-f300-4385-aee4-f31019fabe17 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:56.718019 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438fde3-f300-4385-aee4-f31019fabe17 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:56.718300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438fde3-f300-4385-aee4-f31019fabe17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:56.737075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1059/2060] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:56 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:56.745420 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c438fde3-f300-4385-aee4-f31019fabe17 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:56.745420 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c438fde3-f300-4385-aee4-f31019fabe17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:56.745953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f407cc4-12c4-47ef-beeb-a6a8c743c991 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:56.749326 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9f407cc4-12c4-47ef-beeb-a6a8c743c991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:56.772374 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:56.836630 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1002/2061] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:56.840762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f407cc4-12c4-47ef-beeb-a6a8c743c991 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:56.844782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad54eb-4a67-4d0a-859b-b2329a0fca7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:56.845404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad54eb-4a67-4d0a-859b-b2329a0fca7f admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:56.845684 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad54eb-4a67-4d0a-859b-b2329a0fca7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:56.845957 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad54eb-4a67-4d0a-859b-b2329a0fca7f admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:56.846597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad54eb-4a67-4d0a-859b-b2329a0fca7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:56.848567 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad54eb-4a67-4d0a-859b-b2329a0fca7f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:56.852624 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438fde3-f300-4385-aee4-f31019fabe17 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:56.854256 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c438fde3-f300-4385-aee4-f31019fabe17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:56.861879 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:56.882545 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1060/2062] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:56 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:33:56.888140 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad54eb-4a67-4d0a-859b-b2329a0fca7f admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:56.900995 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad54eb-4a67-4d0a-859b-b2329a0fca7f admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:56.906841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023ba97-57c5-444c-b0db-8e703d295a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:56.907511 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023ba97-57c5-444c-b0db-8e703d295a09 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:56.907799 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023ba97-57c5-444c-b0db-8e703d295a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:56.908140 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023ba97-57c5-444c-b0db-8e703d295a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:56.909063 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023ba97-57c5-444c-b0db-8e703d295a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:56.911050 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023ba97-57c5-444c-b0db-8e703d295a09 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:56.925357 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1061/2063] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:56.927293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023ba97-57c5-444c-b0db-8e703d295a09 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:56.935324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438fde3-f300-4385-aee4-f31019fabe17 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:56.939871 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792b4d9-58a8-4f27-826f-7e728180266f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:56.940101 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023ba97-57c5-444c-b0db-8e703d295a09 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:56.941097 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b4d9-58a8-4f27-826f-7e728180266f service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:56.941572 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b4d9-58a8-4f27-826f-7e728180266f service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:56.941940 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b4d9-58a8-4f27-826f-7e728180266f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:56.944979 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a3a8e-d274-4a13-8d0e-12d751fbb79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:56.945618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a3a8e-d274-4a13-8d0e-12d751fbb79d admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:56.945902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a3a8e-d274-4a13-8d0e-12d751fbb79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:56.946176 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a3a8e-d274-4a13-8d0e-12d751fbb79d admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:56.946837 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a3a8e-d274-4a13-8d0e-12d751fbb79d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:56.948476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a3a8e-d274-4a13-8d0e-12d751fbb79d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:56.954900 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792b4d9-58a8-4f27-826f-7e728180266f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:56.956624 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792b4d9-58a8-4f27-826f-7e728180266f service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:56.967419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1062/2064] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:56.969763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a3a8e-d274-4a13-8d0e-12d751fbb79d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:56.976683 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1003/2065] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:33:56.981498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a3a8e-d274-4a13-8d0e-12d751fbb79d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:56.986787 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d9073-750c-4318-b255-c16729e4be51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:56.991976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d9073-750c-4318-b255-c16729e4be51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:56.991976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d9073-750c-4318-b255-c16729e4be51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:56.991976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d9073-750c-4318-b255-c16729e4be51 admin admin] PATH_INFO: `/v3/projects/bdcd47f8d1ae4b60af7f042c06d0700e/users/5d252eb48d7843048f9626d416b16211/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:57.020115 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d9073-750c-4318-b255-c16729e4be51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdcd47f8d1ae4b60af7f042c06d0700e, user_id=5d252eb48d7843048f9626d416b16211, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:57.022826 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d9073-750c-4318-b255-c16729e4be51 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:57.046288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1063/2066] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:56 2022] PUT /identity/v3/projects/bdcd47f8d1ae4b60af7f042c06d0700e/users/5d252eb48d7843048f9626d416b16211/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:57.058684 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d54ba-ba80-4a2b-8790-0ce1bc68933b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:57.058974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d54ba-ba80-4a2b-8790-0ce1bc68933b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:57.059250 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d54ba-ba80-4a2b-8790-0ce1bc68933b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:57.093568 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-173d54ba-ba80-4a2b-8790-0ce1bc68933b None None] MFA Rules not processed for user `5d252eb48d7843048f9626d416b16211`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:57.094619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-173d54ba-ba80-4a2b-8790-0ce1bc68933b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:57.202225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1004/2067] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:33:57.206613 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d9073-750c-4318-b255-c16729e4be51 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:57.342697 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d54ba-ba80-4a2b-8790-0ce1bc68933b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:57.345386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-173d54ba-ba80-4a2b-8790-0ce1bc68933b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:57.349172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d9073-750c-4318-b255-c16729e4be51 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:57.370303 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbe265-c15b-4433-a2fb-5c5e8cfb301a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:57.370303 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbe265-c15b-4433-a2fb-5c5e8cfb301a admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:57.370303 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbe265-c15b-4433-a2fb-5c5e8cfb301a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:57.370303 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddbe265-c15b-4433-a2fb-5c5e8cfb301a admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:57.372173 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddbe265-c15b-4433-a2fb-5c5e8cfb301a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:57.372173 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddbe265-c15b-4433-a2fb-5c5e8cfb301a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:57.390763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d54ba-ba80-4a2b-8790-0ce1bc68933b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:57.397520 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb6df1-2d68-4136-9f7e-46ec10bd9b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:57.398851 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb6df1-2d68-4136-9f7e-46ec10bd9b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:57.399140 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb6df1-2d68-4136-9f7e-46ec10bd9b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:57.399414 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb6df1-2d68-4136-9f7e-46ec10bd9b3b admin admin] PATH_INFO: `/v3/projects/1f78f3c41743411f8b8395ea6c1357f0` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:57.404195 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb6df1-2d68-4136-9f7e-46ec10bd9b3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f78f3c41743411f8b8395ea6c1357f0)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:57.407622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb6df1-2d68-4136-9f7e-46ec10bd9b3b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:57.423356 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1064/2068] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:33:57 2022] POST /identity/v3/projects => generated 383 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:57.428497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbe265-c15b-4433-a2fb-5c5e8cfb301a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:57.512719 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-30cb6df1-2d68-4136-9f7e-46ec10bd9b3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f78f3c41743411f8b8395ea6c1357f0', 'request_id': 'req-30cb6df1-2d68-4136-9f7e-46ec10bd9b3b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:57.525074 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:57.528552 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-30cb6df1-2d68-4136-9f7e-46ec10bd9b3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f78f3c41743411f8b8395ea6c1357f0', 'request_id': 'req-30cb6df1-2d68-4136-9f7e-46ec10bd9b3b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:57.548520 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-30cb6df1-2d68-4136-9f7e-46ec10bd9b3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f78f3c41743411f8b8395ea6c1357f0', 'request_id': 'req-30cb6df1-2d68-4136-9f7e-46ec10bd9b3b'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:57.576646 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-30cb6df1-2d68-4136-9f7e-46ec10bd9b3b admin admin] The token cache is being invalidate because project 1f78f3c41743411f8b8395ea6c1357f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:57.577231 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-30cb6df1-2d68-4136-9f7e-46ec10bd9b3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:57.580709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddbe265-c15b-4433-a2fb-5c5e8cfb301a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:57.591192 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-30cb6df1-2d68-4136-9f7e-46ec10bd9b3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:57.604997 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1005/2069] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:57 2022] DELETE /identity/v3/projects/1f78f3c41743411f8b8395ea6c1357f0 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:57.606086 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb6df1-2d68-4136-9f7e-46ec10bd9b3b admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:57.610785 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-30cb6df1-2d68-4136-9f7e-46ec10bd9b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:57.679769 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:57.722496 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb6df1-2d68-4136-9f7e-46ec10bd9b3b admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:57.728260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5a777-90f2-4adf-9f71-0cb9731eb0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:57.729165 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5a777-90f2-4adf-9f71-0cb9731eb0ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:57.729578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5a777-90f2-4adf-9f71-0cb9731eb0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:57.730035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5a777-90f2-4adf-9f71-0cb9731eb0ae admin admin] PATH_INFO: `/v3/users/3b190895c0e24daba533151decdc14f7` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:57.731631 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5a777-90f2-4adf-9f71-0cb9731eb0ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b190895c0e24daba533151decdc14f7)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:57.734282 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5a777-90f2-4adf-9f71-0cb9731eb0ae admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:57.744154 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187af62d-1fc9-46a8-9bbf-2d6822766991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:57.744793 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187af62d-1fc9-46a8-9bbf-2d6822766991 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:57.745073 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187af62d-1fc9-46a8-9bbf-2d6822766991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:57.745346 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187af62d-1fc9-46a8-9bbf-2d6822766991 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:57.746327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187af62d-1fc9-46a8-9bbf-2d6822766991 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:57.749010 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187af62d-1fc9-46a8-9bbf-2d6822766991 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:57.781501 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1065/2070] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:33:57 2022] POST /identity/v3/users => generated 298 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:33:57.785022 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187af62d-1fc9-46a8-9bbf-2d6822766991 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:57.789057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-187af62d-1fc9-46a8-9bbf-2d6822766991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:57.810070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187af62d-1fc9-46a8-9bbf-2d6822766991 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:57.817524 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a9fcc-3c9f-4c0d-8f69-7bf38ebae8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:57.817799 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c5e5a777-90f2-4adf-9f71-0cb9731eb0ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b190895c0e24daba533151decdc14f7'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:57.818311 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a9fcc-3c9f-4c0d-8f69-7bf38ebae8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:57.818794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a9fcc-3c9f-4c0d-8f69-7bf38ebae8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:57.819140 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a9fcc-3c9f-4c0d-8f69-7bf38ebae8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:57.820237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a9fcc-3c9f-4c0d-8f69-7bf38ebae8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:57.822402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a9fcc-3c9f-4c0d-8f69-7bf38ebae8e7 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:57.831227 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c5e5a777-90f2-4adf-9f71-0cb9731eb0ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b190895c0e24daba533151decdc14f7'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:57.839222 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1066/2071] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:33:57.852780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a9fcc-3c9f-4c0d-8f69-7bf38ebae8e7 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:57.859890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c5e5a777-90f2-4adf-9f71-0cb9731eb0ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b190895c0e24daba533151decdc14f7'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:57.860860 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c5e5a777-90f2-4adf-9f71-0cb9731eb0ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b190895c0e24daba533151decdc14f7'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:57.866576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a9fcc-3c9f-4c0d-8f69-7bf38ebae8e7 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:57.881520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564481c3-ade8-4db7-8949-8284e43ee00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:57.883737 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564481c3-ade8-4db7-8949-8284e43ee00b admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:57.883737 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564481c3-ade8-4db7-8949-8284e43ee00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:57.883737 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564481c3-ade8-4db7-8949-8284e43ee00b admin admin] PATH_INFO: `/v3/projects/7d35942f930c4ed79efd56176eec81b4/users/75ad613e44db40d486e78d5b4b0ea479/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:57.898519 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c5e5a777-90f2-4adf-9f71-0cb9731eb0ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b190895c0e24daba533151decdc14f7'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:57.902945 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564481c3-ade8-4db7-8949-8284e43ee00b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d35942f930c4ed79efd56176eec81b4, user_id=75ad613e44db40d486e78d5b4b0ea479, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:57.907534 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1006/2072] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:57 2022] DELETE /identity/v3/users/3b190895c0e24daba533151decdc14f7 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:57.908408 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564481c3-ade8-4db7-8949-8284e43ee00b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:57.943354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1067/2073] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:57 2022] PUT /identity/v3/projects/7d35942f930c4ed79efd56176eec81b4/users/75ad613e44db40d486e78d5b4b0ea479/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:57.949680 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5a777-90f2-4adf-9f71-0cb9731eb0ae admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:57.951198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c5e5a777-90f2-4adf-9f71-0cb9731eb0ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:58.076183 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5a777-90f2-4adf-9f71-0cb9731eb0ae admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:58.083847 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:58.193188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37203f09-de2e-4fdf-b5ee-19d57e1e4b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:58.193188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37203f09-de2e-4fdf-b5ee-19d57e1e4b63 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:58.193188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37203f09-de2e-4fdf-b5ee-19d57e1e4b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:58.193188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37203f09-de2e-4fdf-b5ee-19d57e1e4b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:58.193569 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37203f09-de2e-4fdf-b5ee-19d57e1e4b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:58.193569 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37203f09-de2e-4fdf-b5ee-19d57e1e4b63 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:58.206617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1007/2074] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:33:58.211163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564481c3-ade8-4db7-8949-8284e43ee00b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:58.227666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564481c3-ade8-4db7-8949-8284e43ee00b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:58.237659 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545cf45-5233-4c6e-af92-7fcc7a79f5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:58.238515 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545cf45-5233-4c6e-af92-7fcc7a79f5cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:58.238920 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545cf45-5233-4c6e-af92-7fcc7a79f5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:58.239407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545cf45-5233-4c6e-af92-7fcc7a79f5cf admin admin] PATH_INFO: `/v3/projects/7d35942f930c4ed79efd56176eec81b4/users/75ad613e44db40d486e78d5b4b0ea479/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:58.241555 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545cf45-5233-4c6e-af92-7fcc7a79f5cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d35942f930c4ed79efd56176eec81b4, user_id=75ad613e44db40d486e78d5b4b0ea479, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:58.244479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545cf45-5233-4c6e-af92-7fcc7a79f5cf admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:58.270882 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1068/2075] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:33:58 2022] PUT /identity/v3/projects/7d35942f930c4ed79efd56176eec81b4/users/75ad613e44db40d486e78d5b4b0ea479/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:58.272408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b126ca6-35ee-4d18-961e-6bbd718e5591 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:58.273222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b126ca6-35ee-4d18-961e-6bbd718e5591 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:58.273222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b126ca6-35ee-4d18-961e-6bbd718e5591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:58.331362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-8b126ca6-35ee-4d18-961e-6bbd718e5591 None None] MFA Rules not processed for user `75ad613e44db40d486e78d5b4b0ea479`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:33:58.331362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8b126ca6-35ee-4d18-961e-6bbd718e5591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:58.399847 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545cf45-5233-4c6e-af92-7fcc7a79f5cf admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:58.402139 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d545cf45-5233-4c6e-af92-7fcc7a79f5cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:58.515196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1008/2076] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:33:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:33:58.520119 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545cf45-5233-4c6e-af92-7fcc7a79f5cf admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:58.525784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccf9f8-e58b-4fc0-bad9-66556d812e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:58.526701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccf9f8-e58b-4fc0-bad9-66556d812e5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:58.526853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b126ca6-35ee-4d18-961e-6bbd718e5591 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:58.527019 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccf9f8-e58b-4fc0-bad9-66556d812e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:58.527247 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccf9f8-e58b-4fc0-bad9-66556d812e5c admin admin] PATH_INFO: `/v3/projects/7ed97be0bb7943b7a3562cf2412f8b51` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:58.528498 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8b126ca6-35ee-4d18-961e-6bbd718e5591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:58.528661 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccf9f8-e58b-4fc0-bad9-66556d812e5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ed97be0bb7943b7a3562cf2412f8b51)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:58.530615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccf9f8-e58b-4fc0-bad9-66556d812e5c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:58.610541 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b126ca6-35ee-4d18-961e-6bbd718e5591 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:58.619375 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-54ccf9f8-e58b-4fc0-bad9-66556d812e5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ed97be0bb7943b7a3562cf2412f8b51', 'request_id': 'req-54ccf9f8-e58b-4fc0-bad9-66556d812e5c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:58.630733 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-54ccf9f8-e58b-4fc0-bad9-66556d812e5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ed97be0bb7943b7a3562cf2412f8b51', 'request_id': 'req-54ccf9f8-e58b-4fc0-bad9-66556d812e5c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:58.644456 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-54ccf9f8-e58b-4fc0-bad9-66556d812e5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ed97be0bb7943b7a3562cf2412f8b51', 'request_id': 'req-54ccf9f8-e58b-4fc0-bad9-66556d812e5c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:58.668098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-54ccf9f8-e58b-4fc0-bad9-66556d812e5c admin admin] The token cache is being invalidate because project 7ed97be0bb7943b7a3562cf2412f8b51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:33:58.668406 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-54ccf9f8-e58b-4fc0-bad9-66556d812e5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:58.670474 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-54ccf9f8-e58b-4fc0-bad9-66556d812e5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:58.677715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1069/2077] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:33:58 2022] DELETE /identity/v3/projects/7ed97be0bb7943b7a3562cf2412f8b51 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:33:58.680530 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccf9f8-e58b-4fc0-bad9-66556d812e5c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:58.682508 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-54ccf9f8-e58b-4fc0-bad9-66556d812e5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:58.686326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:58.706628 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ec77a-58e7-4fe1-b2ce-bd5dc3b67edf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:58.706628 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ec77a-58e7-4fe1-b2ce-bd5dc3b67edf service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:58.706628 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ec77a-58e7-4fe1-b2ce-bd5dc3b67edf service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:58.707071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ec77a-58e7-4fe1-b2ce-bd5dc3b67edf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:58.712665 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d25ec77a-58e7-4fe1-b2ce-bd5dc3b67edf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:58.721212 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:33:58.832338 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccf9f8-e58b-4fc0-bad9-66556d812e5c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:58.846870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c057041-ec04-4c08-9744-7111180824ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:58.849355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c057041-ec04-4c08-9744-7111180824ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:58.849714 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c057041-ec04-4c08-9744-7111180824ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:58.851197 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c057041-ec04-4c08-9744-7111180824ee admin admin] PATH_INFO: `/v3/users/51b4793370d04c3d941c8c351e60c078` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:58.857196 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c057041-ec04-4c08-9744-7111180824ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=51b4793370d04c3d941c8c351e60c078)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:58.863050 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c057041-ec04-4c08-9744-7111180824ee admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:58.961443 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ec77a-58e7-4fe1-b2ce-bd5dc3b67edf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:58.963479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ec77a-58e7-4fe1-b2ce-bd5dc3b67edf service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:58.977318 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1009/2078] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:58 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:33:58.990316 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1c057041-ec04-4c08-9744-7111180824ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51b4793370d04c3d941c8c351e60c078'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:59.017779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1c057041-ec04-4c08-9744-7111180824ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51b4793370d04c3d941c8c351e60c078'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:59.031302 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1c057041-ec04-4c08-9744-7111180824ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51b4793370d04c3d941c8c351e60c078'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:59.032232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1c057041-ec04-4c08-9744-7111180824ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51b4793370d04c3d941c8c351e60c078'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:59.033520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1c057041-ec04-4c08-9744-7111180824ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51b4793370d04c3d941c8c351e60c078'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:33:59.092709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1070/2079] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:33:58 2022] DELETE /identity/v3/users/51b4793370d04c3d941c8c351e60c078 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:33:59.147906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ec77a-58e7-4fe1-b2ce-bd5dc3b67edf service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:33:59.150048 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d25ec77a-58e7-4fe1-b2ce-bd5dc3b67edf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:59.162807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ec77a-58e7-4fe1-b2ce-bd5dc3b67edf service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:33:59.233391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e54534f-91e2-450c-a052-fe9cddea2f0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:33:59.234282 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54534f-91e2-450c-a052-fe9cddea2f0e service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:33:59.235159 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54534f-91e2-450c-a052-fe9cddea2f0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:33:59.235159 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54534f-91e2-450c-a052-fe9cddea2f0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:33:59.237773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1e54534f-91e2-450c-a052-fe9cddea2f0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:33:59.338100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e54534f-91e2-450c-a052-fe9cddea2f0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:33:59.339908 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e54534f-91e2-450c-a052-fe9cddea2f0e service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:33:59.351168 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1010/2080] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:33:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:00.164874 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c057041-ec04-4c08-9744-7111180824ee admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:00.168286 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1c057041-ec04-4c08-9744-7111180824ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:00.245768 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e54534f-91e2-450c-a052-fe9cddea2f0e service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:00.247545 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1e54534f-91e2-450c-a052-fe9cddea2f0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:00.364529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c057041-ec04-4c08-9744-7111180824ee admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:00.373602 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14edf293-eee6-474f-bdfc-75044deb9712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:00.373602 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14edf293-eee6-474f-bdfc-75044deb9712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:00.373939 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14edf293-eee6-474f-bdfc-75044deb9712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:00.374627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14edf293-eee6-474f-bdfc-75044deb9712 admin admin] PATH_INFO: `/v3/users/ea4a04348d1d41039020d277b6c1c98c` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:00.376082 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14edf293-eee6-474f-bdfc-75044deb9712 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea4a04348d1d41039020d277b6c1c98c)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:00.376439 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:00.378045 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14edf293-eee6-474f-bdfc-75044deb9712 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:00.418776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e54534f-91e2-450c-a052-fe9cddea2f0e service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:00.425147 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0aea1-7d39-494b-8d77-3b33f9240dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:00.425968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0aea1-7d39-494b-8d77-3b33f9240dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:00.426546 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0aea1-7d39-494b-8d77-3b33f9240dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:00.427980 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0aea1-7d39-494b-8d77-3b33f9240dc8 admin admin] PATH_INFO: `/v3/projects/60b7ec2b97404072a76a0570180f4acb` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:00.429265 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0aea1-7d39-494b-8d77-3b33f9240dc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b7ec2b97404072a76a0570180f4acb)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:00.431719 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0aea1-7d39-494b-8d77-3b33f9240dc8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:00.450851 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:00.467306 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-14edf293-eee6-474f-bdfc-75044deb9712 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea4a04348d1d41039020d277b6c1c98c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:00.479122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-14edf293-eee6-474f-bdfc-75044deb9712 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea4a04348d1d41039020d277b6c1c98c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:00.494044 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-14edf293-eee6-474f-bdfc-75044deb9712 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea4a04348d1d41039020d277b6c1c98c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:00.495449 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-14edf293-eee6-474f-bdfc-75044deb9712 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea4a04348d1d41039020d277b6c1c98c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:00.497621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-14edf293-eee6-474f-bdfc-75044deb9712 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea4a04348d1d41039020d277b6c1c98c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:00.534576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1071/2081] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:00 2022] DELETE /identity/v3/users/ea4a04348d1d41039020d277b6c1c98c => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:00.540157 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6528ad-dc61-47fb-a55f-f31c63a9cae2 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:00.540451 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6528ad-dc61-47fb-a55f-f31c63a9cae2 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:00.540756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6528ad-dc61-47fb-a55f-f31c63a9cae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:00.574635 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-9e6528ad-dc61-47fb-a55f-f31c63a9cae2 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:00.576171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9e6528ad-dc61-47fb-a55f-f31c63a9cae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:00.576524 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5bb0aea1-7d39-494b-8d77-3b33f9240dc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b7ec2b97404072a76a0570180f4acb', 'request_id': 'req-5bb0aea1-7d39-494b-8d77-3b33f9240dc8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:00.583526 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5bb0aea1-7d39-494b-8d77-3b33f9240dc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b7ec2b97404072a76a0570180f4acb', 'request_id': 'req-5bb0aea1-7d39-494b-8d77-3b33f9240dc8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:00.595009 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5bb0aea1-7d39-494b-8d77-3b33f9240dc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b7ec2b97404072a76a0570180f4acb', 'request_id': 'req-5bb0aea1-7d39-494b-8d77-3b33f9240dc8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:00.610147 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5bb0aea1-7d39-494b-8d77-3b33f9240dc8 admin admin] The token cache is being invalidate because project 60b7ec2b97404072a76a0570180f4acb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:00.610524 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5bb0aea1-7d39-494b-8d77-3b33f9240dc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:00.613660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5bb0aea1-7d39-494b-8d77-3b33f9240dc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:00.615479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1011/2082] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:00 2022] DELETE /identity/v3/projects/60b7ec2b97404072a76a0570180f4acb => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:00.621675 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854431c-203f-4af6-8290-ea8d8a51e48d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:00.622472 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854431c-203f-4af6-8290-ea8d8a51e48d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:00.623087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854431c-203f-4af6-8290-ea8d8a51e48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:00.650866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c854431c-203f-4af6-8290-ea8d8a51e48d None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:00.652369 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c854431c-203f-4af6-8290-ea8d8a51e48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:00.654129 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1072/2083] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:00.664096 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6528ad-dc61-47fb-a55f-f31c63a9cae2 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:00.665682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1012/2084] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:34:00.665826 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9e6528ad-dc61-47fb-a55f-f31c63a9cae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:00.673992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858759d-51ad-4091-891d-5741d17111cd None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:00.674274 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858759d-51ad-4091-891d-5741d17111cd None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:00.674546 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858759d-51ad-4091-891d-5741d17111cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:00.700739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c858759d-51ad-4091-891d-5741d17111cd None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:00.702086 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c858759d-51ad-4091-891d-5741d17111cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:00.714912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1013/2085] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:34:00.714912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858759d-51ad-4091-891d-5741d17111cd None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:00.718636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c858759d-51ad-4091-891d-5741d17111cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:00.732290 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858759d-51ad-4091-891d-5741d17111cd None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:00.739396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3d513-755f-4ba2-82c0-6934373c3b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:00.740027 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3d513-755f-4ba2-82c0-6934373c3b52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:00.740310 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3d513-755f-4ba2-82c0-6934373c3b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:00.740587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3d513-755f-4ba2-82c0-6934373c3b52 admin admin] PATH_INFO: `/v3/users/fe0a7965d47449f8ab7683832e2ecfee` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:00.742416 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3d513-755f-4ba2-82c0-6934373c3b52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe0a7965d47449f8ab7683832e2ecfee)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:00.744591 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3d513-755f-4ba2-82c0-6934373c3b52 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:00.763789 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6528ad-dc61-47fb-a55f-f31c63a9cae2 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:00.769825 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079419ee-18b2-4a4b-9dac-69fd6f1e674f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:00.770549 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079419ee-18b2-4a4b-9dac-69fd6f1e674f service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:00.770835 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079419ee-18b2-4a4b-9dac-69fd6f1e674f service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:00.771176 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079419ee-18b2-4a4b-9dac-69fd6f1e674f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:00.782180 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079419ee-18b2-4a4b-9dac-69fd6f1e674f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:00.783701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079419ee-18b2-4a4b-9dac-69fd6f1e674f service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:00.797373 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1073/2086] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:00.798572 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079419ee-18b2-4a4b-9dac-69fd6f1e674f service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:00.806933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3ff3d513-755f-4ba2-82c0-6934373c3b52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe0a7965d47449f8ab7683832e2ecfee'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:00.809922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079419ee-18b2-4a4b-9dac-69fd6f1e674f service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:00.814461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3ff3d513-755f-4ba2-82c0-6934373c3b52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe0a7965d47449f8ab7683832e2ecfee'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:00.815271 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3707bc-a831-4bad-bfc0-30c590f98473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:00.815903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3707bc-a831-4bad-bfc0-30c590f98473 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:00.816225 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3707bc-a831-4bad-bfc0-30c590f98473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:00.816540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3707bc-a831-4bad-bfc0-30c590f98473 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:00.817149 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3707bc-a831-4bad-bfc0-30c590f98473 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:34:00.817742 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3707bc-a831-4bad-bfc0-30c590f98473 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:00.819699 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3707bc-a831-4bad-bfc0-30c590f98473 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:00.825092 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3ff3d513-755f-4ba2-82c0-6934373c3b52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe0a7965d47449f8ab7683832e2ecfee'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:00.825763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3ff3d513-755f-4ba2-82c0-6934373c3b52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe0a7965d47449f8ab7683832e2ecfee'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:00.833256 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1074/2087] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:34:00.837546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3707bc-a831-4bad-bfc0-30c590f98473 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:00.839513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ac3707bc-a831-4bad-bfc0-30c590f98473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:00.844428 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3ff3d513-755f-4ba2-82c0-6934373c3b52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe0a7965d47449f8ab7683832e2ecfee'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:00.853070 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1014/2088] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:00 2022] DELETE /identity/v3/users/fe0a7965d47449f8ab7683832e2ecfee => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:00.859525 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ade64a-66aa-4d49-879d-2041c9234c5a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:00.859525 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ade64a-66aa-4d49-879d-2041c9234c5a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:00.859525 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ade64a-66aa-4d49-879d-2041c9234c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:00.874531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3707bc-a831-4bad-bfc0-30c590f98473 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:00.887359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-98ade64a-66aa-4d49-879d-2041c9234c5a None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:00.888842 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-98ade64a-66aa-4d49-879d-2041c9234c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:00.940296 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:00.941763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:00.985380 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55587031-f16a-4fca-9e1b-32d4947def20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:00.986040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55587031-f16a-4fca-9e1b-32d4947def20 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:00.986908 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55587031-f16a-4fca-9e1b-32d4947def20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:00.987187 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55587031-f16a-4fca-9e1b-32d4947def20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:00.987843 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55587031-f16a-4fca-9e1b-32d4947def20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:00.989806 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55587031-f16a-4fca-9e1b-32d4947def20 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:01.010200 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1015/2089] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:01.020224 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ade64a-66aa-4d49-879d-2041c9234c5a None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:01.022808 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-98ade64a-66aa-4d49-879d-2041c9234c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:01.034759 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1075/2090] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:00 2022] POST /identity/v3/projects => generated 365 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:01.037495 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55587031-f16a-4fca-9e1b-32d4947def20 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:01.118385 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ade64a-66aa-4d49-879d-2041c9234c5a None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:01.184118 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55587031-f16a-4fca-9e1b-32d4947def20 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:01.188198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6607f1-153d-4d6a-a1c7-6328076fb902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:01.189531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6607f1-153d-4d6a-a1c7-6328076fb902 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:01.189824 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6607f1-153d-4d6a-a1c7-6328076fb902 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:01.190276 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6607f1-153d-4d6a-a1c7-6328076fb902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:01.195234 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2ab74-e6db-498a-96f8-8cbfead3fa1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:01.196380 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2ab74-e6db-498a-96f8-8cbfead3fa1b admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:01.196927 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2ab74-e6db-498a-96f8-8cbfead3fa1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:01.197528 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2ab74-e6db-498a-96f8-8cbfead3fa1b admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:01.198366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2ab74-e6db-498a-96f8-8cbfead3fa1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:01.202283 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e2ab74-e6db-498a-96f8-8cbfead3fa1b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:01.205673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6607f1-153d-4d6a-a1c7-6328076fb902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:01.208587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6607f1-153d-4d6a-a1c7-6328076fb902 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:01.236637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1016/2091] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:01.244518 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1076/2092] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:01 2022] POST /identity/v3/users => generated 283 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:01.251472 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6607f1-153d-4d6a-a1c7-6328076fb902 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:01.262088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2ab74-e6db-498a-96f8-8cbfead3fa1b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:01.264929 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-72e2ab74-e6db-498a-96f8-8cbfead3fa1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:01.285158 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2ab74-e6db-498a-96f8-8cbfead3fa1b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:01.287369 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6607f1-153d-4d6a-a1c7-6328076fb902 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:01.294428 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67da2b1-6752-4b38-963a-9337ebf82891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:01.295329 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67da2b1-6752-4b38-963a-9337ebf82891 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:01.295567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67da2b1-6752-4b38-963a-9337ebf82891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:01.295720 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67da2b1-6752-4b38-963a-9337ebf82891 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:01.296329 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67da2b1-6752-4b38-963a-9337ebf82891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:01.298134 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67da2b1-6752-4b38-963a-9337ebf82891 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:01.300850 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8eb34-dd23-4d52-bcae-b3dbf876a998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:01.305455 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8eb34-dd23-4d52-bcae-b3dbf876a998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:01.305911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8eb34-dd23-4d52-bcae-b3dbf876a998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:01.306477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8eb34-dd23-4d52-bcae-b3dbf876a998 admin admin] PATH_INFO: `/v3/projects/70334af7f020436cafe97073b94c2a7e` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:01.310564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1017/2093] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:01.313045 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8eb34-dd23-4d52-bcae-b3dbf876a998 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70334af7f020436cafe97073b94c2a7e)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:01.314986 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67da2b1-6752-4b38-963a-9337ebf82891 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:01.319518 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8eb34-dd23-4d52-bcae-b3dbf876a998 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:01.331372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67da2b1-6752-4b38-963a-9337ebf82891 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:01.339316 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f293605-51a2-4a84-9034-a11e5297af75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:01.339995 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f293605-51a2-4a84-9034-a11e5297af75 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:01.340286 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f293605-51a2-4a84-9034-a11e5297af75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:01.340592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f293605-51a2-4a84-9034-a11e5297af75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:01.341397 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f293605-51a2-4a84-9034-a11e5297af75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:01.343647 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f293605-51a2-4a84-9034-a11e5297af75 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:01.359790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1018/2094] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:01.361023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f293605-51a2-4a84-9034-a11e5297af75 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:01.377750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f293605-51a2-4a84-9034-a11e5297af75 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:01.404117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-eba8eb34-dd23-4d52-bcae-b3dbf876a998 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70334af7f020436cafe97073b94c2a7e', 'request_id': 'req-eba8eb34-dd23-4d52-bcae-b3dbf876a998'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:01.416554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-eba8eb34-dd23-4d52-bcae-b3dbf876a998 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70334af7f020436cafe97073b94c2a7e', 'request_id': 'req-eba8eb34-dd23-4d52-bcae-b3dbf876a998'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:01.427312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-eba8eb34-dd23-4d52-bcae-b3dbf876a998 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70334af7f020436cafe97073b94c2a7e', 'request_id': 'req-eba8eb34-dd23-4d52-bcae-b3dbf876a998'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:01.454186 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-eba8eb34-dd23-4d52-bcae-b3dbf876a998 admin admin] The token cache is being invalidate because project 70334af7f020436cafe97073b94c2a7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:01.454438 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-eba8eb34-dd23-4d52-bcae-b3dbf876a998 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:01.457740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-eba8eb34-dd23-4d52-bcae-b3dbf876a998 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:01.459310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1077/2095] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:01 2022] DELETE /identity/v3/projects/70334af7f020436cafe97073b94c2a7e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:01.464130 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8eb34-dd23-4d52-bcae-b3dbf876a998 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:01.466591 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-eba8eb34-dd23-4d52-bcae-b3dbf876a998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:01.489521 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2100af6-d657-452b-ac3e-b87031b81bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:01.490225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2100af6-d657-452b-ac3e-b87031b81bde admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:01.490513 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2100af6-d657-452b-ac3e-b87031b81bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:01.490797 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2100af6-d657-452b-ac3e-b87031b81bde admin admin] PATH_INFO: `/v3/projects/d26c88c225c542eaa1cfb2c459e292f6/users/dc5e45f9539048c4a85a649d486d1ba3/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:01.516685 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2100af6-d657-452b-ac3e-b87031b81bde admin admin] RBAC: Authorizing `identity:create_grant(project_id=d26c88c225c542eaa1cfb2c459e292f6, user_id=dc5e45f9539048c4a85a649d486d1ba3, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:01.520667 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2100af6-d657-452b-ac3e-b87031b81bde admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:01.528461 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:01.535603 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:01.548964 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1019/2096] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:01 2022] PUT /identity/v3/projects/d26c88c225c542eaa1cfb2c459e292f6/users/dc5e45f9539048c4a85a649d486d1ba3/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:34:01.566996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a47acc-8ef0-43ed-8b01-a3e9229e4564 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:01.567282 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a47acc-8ef0-43ed-8b01-a3e9229e4564 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:01.567558 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a47acc-8ef0-43ed-8b01-a3e9229e4564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:01.570235 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8eb34-dd23-4d52-bcae-b3dbf876a998 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:01.619854 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d4a47acc-8ef0-43ed-8b01-a3e9229e4564 None None] MFA Rules not processed for user `dc5e45f9539048c4a85a649d486d1ba3`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:01.621118 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d4a47acc-8ef0-43ed-8b01-a3e9229e4564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:01.686241 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d40fcc-1c04-42bd-9e4a-b240f777fc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:01.687317 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d40fcc-1c04-42bd-9e4a-b240f777fc2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:01.687858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d40fcc-1c04-42bd-9e4a-b240f777fc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:01.688476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d40fcc-1c04-42bd-9e4a-b240f777fc2c admin admin] PATH_INFO: `/v3/users/e7c0f550253244f38a0d1741791ecb88` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:01.690157 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d40fcc-1c04-42bd-9e4a-b240f777fc2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7c0f550253244f38a0d1741791ecb88)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:01.696166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d40fcc-1c04-42bd-9e4a-b240f777fc2c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:01.731776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1020/2097] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:34:01.733443 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a47acc-8ef0-43ed-8b01-a3e9229e4564 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:01.747030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d4a47acc-8ef0-43ed-8b01-a3e9229e4564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:01.772586 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a47acc-8ef0-43ed-8b01-a3e9229e4564 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:01.778648 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e76510-a70c-4426-a0f5-e75d7c462150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:01.779142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e76510-a70c-4426-a0f5-e75d7c462150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:01.779437 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e76510-a70c-4426-a0f5-e75d7c462150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:01.779735 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e76510-a70c-4426-a0f5-e75d7c462150 admin admin] PATH_INFO: `/v3/projects/d898a6b1c9f94a17bbb98af4fe4e7c3e` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:01.780395 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e4d40fcc-1c04-42bd-9e4a-b240f777fc2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7c0f550253244f38a0d1741791ecb88'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:01.781158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e76510-a70c-4426-a0f5-e75d7c462150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d898a6b1c9f94a17bbb98af4fe4e7c3e)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:01.784353 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e76510-a70c-4426-a0f5-e75d7c462150 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:01.789829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e4d40fcc-1c04-42bd-9e4a-b240f777fc2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7c0f550253244f38a0d1741791ecb88'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:01.815557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e4d40fcc-1c04-42bd-9e4a-b240f777fc2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7c0f550253244f38a0d1741791ecb88'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:01.815557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e4d40fcc-1c04-42bd-9e4a-b240f777fc2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7c0f550253244f38a0d1741791ecb88'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:01.815557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e4d40fcc-1c04-42bd-9e4a-b240f777fc2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7c0f550253244f38a0d1741791ecb88'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:01.840626 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1078/2098] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:01 2022] DELETE /identity/v3/users/e7c0f550253244f38a0d1741791ecb88 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:01.840626 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d40fcc-1c04-42bd-9e4a-b240f777fc2c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:01.840626 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e4d40fcc-1c04-42bd-9e4a-b240f777fc2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:01.870450 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c0e76510-a70c-4426-a0f5-e75d7c462150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd898a6b1c9f94a17bbb98af4fe4e7c3e', 'request_id': 'req-c0e76510-a70c-4426-a0f5-e75d7c462150'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:01.884257 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c0e76510-a70c-4426-a0f5-e75d7c462150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd898a6b1c9f94a17bbb98af4fe4e7c3e', 'request_id': 'req-c0e76510-a70c-4426-a0f5-e75d7c462150'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:01.891483 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c0e76510-a70c-4426-a0f5-e75d7c462150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd898a6b1c9f94a17bbb98af4fe4e7c3e', 'request_id': 'req-c0e76510-a70c-4426-a0f5-e75d7c462150'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:01.907925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c0e76510-a70c-4426-a0f5-e75d7c462150 admin admin] The token cache is being invalidate because project d898a6b1c9f94a17bbb98af4fe4e7c3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:01.908587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c0e76510-a70c-4426-a0f5-e75d7c462150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:01.909574 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-c0e76510-a70c-4426-a0f5-e75d7c462150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:01.911883 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1021/2099] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:01 2022] DELETE /identity/v3/projects/d898a6b1c9f94a17bbb98af4fe4e7c3e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:01.917066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e76510-a70c-4426-a0f5-e75d7c462150 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:01.919545 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c0e76510-a70c-4426-a0f5-e75d7c462150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:01.926732 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d40fcc-1c04-42bd-9e4a-b240f777fc2c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:01.956427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:02.011749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56f470-56e8-4fb3-8396-5090712aa6e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:02.012750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56f470-56e8-4fb3-8396-5090712aa6e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:02.013254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56f470-56e8-4fb3-8396-5090712aa6e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:02.013815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56f470-56e8-4fb3-8396-5090712aa6e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:02.016287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-eb56f470-56e8-4fb3-8396-5090712aa6e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:02.046100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e76510-a70c-4426-a0f5-e75d7c462150 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:02.062427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f055d73-f4c6-422b-8c69-0831a2b8a45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:02.063176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f055d73-f4c6-422b-8c69-0831a2b8a45c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:02.063459 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f055d73-f4c6-422b-8c69-0831a2b8a45c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:02.063731 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f055d73-f4c6-422b-8c69-0831a2b8a45c admin admin] PATH_INFO: `/v3/users/5a2330db9585492e9520143978bc3702` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:02.066806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f055d73-f4c6-422b-8c69-0831a2b8a45c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a2330db9585492e9520143978bc3702)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:02.069030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f055d73-f4c6-422b-8c69-0831a2b8a45c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:02.099067 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:02.128902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56f470-56e8-4fb3-8396-5090712aa6e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:02.131194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56f470-56e8-4fb3-8396-5090712aa6e9 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:02.147703 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1079/2100] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:02.162618 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2f055d73-f4c6-422b-8c69-0831a2b8a45c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a2330db9585492e9520143978bc3702'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:02.171633 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2f055d73-f4c6-422b-8c69-0831a2b8a45c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a2330db9585492e9520143978bc3702'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:02.186213 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2f055d73-f4c6-422b-8c69-0831a2b8a45c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a2330db9585492e9520143978bc3702'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:02.187221 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2f055d73-f4c6-422b-8c69-0831a2b8a45c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a2330db9585492e9520143978bc3702'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:02.204004 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-2f055d73-f4c6-422b-8c69-0831a2b8a45c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a2330db9585492e9520143978bc3702'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:02.212159 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1022/2101] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:01 2022] DELETE /identity/v3/users/5a2330db9585492e9520143978bc3702 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:02.256439 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56f470-56e8-4fb3-8396-5090712aa6e9 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:02.258710 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-eb56f470-56e8-4fb3-8396-5090712aa6e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:02.295521 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56f470-56e8-4fb3-8396-5090712aa6e9 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:02.368947 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f055d73-f4c6-422b-8c69-0831a2b8a45c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:02.370520 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2f055d73-f4c6-422b-8c69-0831a2b8a45c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:02.392881 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77faa34c-6163-4d24-8d44-4c9a7eba7ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:02.393544 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faa34c-6163-4d24-8d44-4c9a7eba7ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:02.393832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faa34c-6163-4d24-8d44-4c9a7eba7ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:02.394120 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faa34c-6163-4d24-8d44-4c9a7eba7ff9 admin admin] PATH_INFO: `/v3/projects/c106941afde44402a5bf2cf04c543eef` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:02.396112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77faa34c-6163-4d24-8d44-4c9a7eba7ff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c106941afde44402a5bf2cf04c543eef)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:02.400037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77faa34c-6163-4d24-8d44-4c9a7eba7ff9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:02.467617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f055d73-f4c6-422b-8c69-0831a2b8a45c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:02.474474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d2fea-e7fc-450e-bdb6-6a624730f54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:02.475437 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d2fea-e7fc-450e-bdb6-6a624730f54a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:02.476811 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d2fea-e7fc-450e-bdb6-6a624730f54a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:02.478135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d2fea-e7fc-450e-bdb6-6a624730f54a admin admin] PATH_INFO: `/v3/users/5d252eb48d7843048f9626d416b16211` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:02.480110 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d2fea-e7fc-450e-bdb6-6a624730f54a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d252eb48d7843048f9626d416b16211)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:02.482308 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d2fea-e7fc-450e-bdb6-6a624730f54a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:02.502553 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-77faa34c-6163-4d24-8d44-4c9a7eba7ff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c106941afde44402a5bf2cf04c543eef', 'request_id': 'req-77faa34c-6163-4d24-8d44-4c9a7eba7ff9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:02.515019 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-77faa34c-6163-4d24-8d44-4c9a7eba7ff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c106941afde44402a5bf2cf04c543eef', 'request_id': 'req-77faa34c-6163-4d24-8d44-4c9a7eba7ff9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:02.528937 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-77faa34c-6163-4d24-8d44-4c9a7eba7ff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c106941afde44402a5bf2cf04c543eef', 'request_id': 'req-77faa34c-6163-4d24-8d44-4c9a7eba7ff9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:02.554744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-77faa34c-6163-4d24-8d44-4c9a7eba7ff9 admin admin] The token cache is being invalidate because project c106941afde44402a5bf2cf04c543eef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:02.555160 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-77faa34c-6163-4d24-8d44-4c9a7eba7ff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:02.555969 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-77faa34c-6163-4d24-8d44-4c9a7eba7ff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:02.561073 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1080/2102] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:02 2022] DELETE /identity/v3/projects/c106941afde44402a5bf2cf04c543eef => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:02.564271 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9fa3c-4f2d-41a6-8e28-be567e8d74d5 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:02.565129 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9fa3c-4f2d-41a6-8e28-be567e8d74d5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:02.565419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9fa3c-4f2d-41a6-8e28-be567e8d74d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:02.585228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-144d2fea-e7fc-450e-bdb6-6a624730f54a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d252eb48d7843048f9626d416b16211'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:02.585339 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:02.592608 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-144d2fea-e7fc-450e-bdb6-6a624730f54a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d252eb48d7843048f9626d416b16211'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:02.602248 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-7af9fa3c-4f2d-41a6-8e28-be567e8d74d5 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:02.603605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7af9fa3c-4f2d-41a6-8e28-be567e8d74d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:02.614502 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-144d2fea-e7fc-450e-bdb6-6a624730f54a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d252eb48d7843048f9626d416b16211'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:02.615720 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-144d2fea-e7fc-450e-bdb6-6a624730f54a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d252eb48d7843048f9626d416b16211'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:02.653371 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-144d2fea-e7fc-450e-bdb6-6a624730f54a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d252eb48d7843048f9626d416b16211'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:02.674098 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1023/2103] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:02 2022] DELETE /identity/v3/users/5d252eb48d7843048f9626d416b16211 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:02.675069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d2fea-e7fc-450e-bdb6-6a624730f54a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:02.679531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-144d2fea-e7fc-450e-bdb6-6a624730f54a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:02.700764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:02.707626 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1081/2104] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:02.709574 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd289f-c5d0-48d6-9cb6-04d07082abb7 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:02.709872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd289f-c5d0-48d6-9cb6-04d07082abb7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:02.710176 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd289f-c5d0-48d6-9cb6-04d07082abb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:02.732373 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-1fdd289f-c5d0-48d6-9cb6-04d07082abb7 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:02.733850 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1fdd289f-c5d0-48d6-9cb6-04d07082abb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:02.768961 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d2fea-e7fc-450e-bdb6-6a624730f54a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:02.774355 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102b402-8bf6-4921-8e76-8c6353d518be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:02.775016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102b402-8bf6-4921-8e76-8c6353d518be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:02.775287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102b402-8bf6-4921-8e76-8c6353d518be admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:02.775559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102b402-8bf6-4921-8e76-8c6353d518be admin admin] PATH_INFO: `/v3/projects/b6e47b12cfc3402181784fc901d859d5` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:02.776708 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102b402-8bf6-4921-8e76-8c6353d518be admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6e47b12cfc3402181784fc901d859d5)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:02.778610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102b402-8bf6-4921-8e76-8c6353d518be admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:02.811274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1082/2105] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:02.812630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154dcffd-288e-4cbe-917f-41825e1e6a2e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:02.812914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154dcffd-288e-4cbe-917f-41825e1e6a2e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:02.813187 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154dcffd-288e-4cbe-917f-41825e1e6a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:02.830796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e102b402-8bf6-4921-8e76-8c6353d518be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6e47b12cfc3402181784fc901d859d5', 'request_id': 'req-e102b402-8bf6-4921-8e76-8c6353d518be'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:02.833613 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-154dcffd-288e-4cbe-917f-41825e1e6a2e None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:02.834984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-154dcffd-288e-4cbe-917f-41825e1e6a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:02.836828 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e102b402-8bf6-4921-8e76-8c6353d518be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6e47b12cfc3402181784fc901d859d5', 'request_id': 'req-e102b402-8bf6-4921-8e76-8c6353d518be'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:02.844277 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e102b402-8bf6-4921-8e76-8c6353d518be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6e47b12cfc3402181784fc901d859d5', 'request_id': 'req-e102b402-8bf6-4921-8e76-8c6353d518be'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:02.858795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e102b402-8bf6-4921-8e76-8c6353d518be admin admin] The token cache is being invalidate because project b6e47b12cfc3402181784fc901d859d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:02.859088 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e102b402-8bf6-4921-8e76-8c6353d518be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:02.860863 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e102b402-8bf6-4921-8e76-8c6353d518be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:02.863317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1024/2106] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:02 2022] DELETE /identity/v3/projects/b6e47b12cfc3402181784fc901d859d5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:02.864174 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102b402-8bf6-4921-8e76-8c6353d518be admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:02.865857 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e102b402-8bf6-4921-8e76-8c6353d518be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:02.922922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1083/2107] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:02.922922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d3b64-1220-4b03-9c7e-6a169d672bc5 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:02.922922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d3b64-1220-4b03-9c7e-6a169d672bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:02.922922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d3b64-1220-4b03-9c7e-6a169d672bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:02.949143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102b402-8bf6-4921-8e76-8c6353d518be admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:02.949893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-dd6d3b64-1220-4b03-9c7e-6a169d672bc5 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:02.951447 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-dd6d3b64-1220-4b03-9c7e-6a169d672bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:02.953809 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b510464-854f-4611-a2f9-9e06e8feac18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:02.954476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b510464-854f-4611-a2f9-9e06e8feac18 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:02.954743 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b510464-854f-4611-a2f9-9e06e8feac18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:02.955244 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b510464-854f-4611-a2f9-9e06e8feac18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:02.961696 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9b510464-854f-4611-a2f9-9e06e8feac18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:02.961880 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1084/2108] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:02.962771 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d3b64-1220-4b03-9c7e-6a169d672bc5 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:02.974004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d3b64-1220-4b03-9c7e-6a169d672bc5 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:02.979885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b510464-854f-4611-a2f9-9e06e8feac18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:02.980974 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32886416-593c-4918-b46c-bdd215c4ea7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:02.981524 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b510464-854f-4611-a2f9-9e06e8feac18 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:02.982221 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32886416-593c-4918-b46c-bdd215c4ea7e admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:02.982829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32886416-593c-4918-b46c-bdd215c4ea7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:02.983369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32886416-593c-4918-b46c-bdd215c4ea7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:02.984240 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32886416-593c-4918-b46c-bdd215c4ea7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:34:02.984913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32886416-593c-4918-b46c-bdd215c4ea7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:02.987660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32886416-593c-4918-b46c-bdd215c4ea7e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:02.996001 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1025/2109] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:03.001443 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c5e30-755d-463c-a512-a51a589a44e4 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:03.003615 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c5e30-755d-463c-a512-a51a589a44e4 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:03.003902 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c5e30-755d-463c-a512-a51a589a44e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:03.016202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1085/2110] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:34:03.023193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32886416-593c-4918-b46c-bdd215c4ea7e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:03.038077 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32886416-593c-4918-b46c-bdd215c4ea7e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:03.042953 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d36c5e30-755d-463c-a512-a51a589a44e4 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:03.042953 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d36c5e30-755d-463c-a512-a51a589a44e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:03.053201 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdefcfc-8987-4c54-94bb-9d53603bb248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:03.054654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdefcfc-8987-4c54-94bb-9d53603bb248 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:03.055126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdefcfc-8987-4c54-94bb-9d53603bb248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:03.055680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdefcfc-8987-4c54-94bb-9d53603bb248 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:03.057051 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdefcfc-8987-4c54-94bb-9d53603bb248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:03.060206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdefcfc-8987-4c54-94bb-9d53603bb248 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:03.060492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1026/2111] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:03.069938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c5e30-755d-463c-a512-a51a589a44e4 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:03.091865 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c5e30-755d-463c-a512-a51a589a44e4 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:03.109795 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1086/2112] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:03 2022] POST /identity/v3/projects => generated 375 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:03.110207 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb82956-6ae5-4d1f-ba87-91cf75b930df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:03.117480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdefcfc-8987-4c54-94bb-9d53603bb248 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:03.121970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb82956-6ae5-4d1f-ba87-91cf75b930df admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:03.122339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb82956-6ae5-4d1f-ba87-91cf75b930df admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:03.122748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb82956-6ae5-4d1f-ba87-91cf75b930df admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:03.123702 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb82956-6ae5-4d1f-ba87-91cf75b930df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:34:03.129769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb82956-6ae5-4d1f-ba87-91cf75b930df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:03.130418 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:03.132519 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb82956-6ae5-4d1f-ba87-91cf75b930df admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:03.149665 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1027/2113] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:34:03.156416 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb82956-6ae5-4d1f-ba87-91cf75b930df admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:03.186048 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb82956-6ae5-4d1f-ba87-91cf75b930df admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:03.233478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:03.269485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdefcfc-8987-4c54-94bb-9d53603bb248 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:03.276067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ee802-977d-4332-a9ae-79e533e9fbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:03.276654 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ee802-977d-4332-a9ae-79e533e9fbf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:03.276936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ee802-977d-4332-a9ae-79e533e9fbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:03.277211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ee802-977d-4332-a9ae-79e533e9fbf8 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:03.277892 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ee802-977d-4332-a9ae-79e533e9fbf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:03.279836 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ee802-977d-4332-a9ae-79e533e9fbf8 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:03.318938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b53436-50cd-44c1-b346-f7052f58c2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:03.319773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b53436-50cd-44c1-b346-f7052f58c2be admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:03.320165 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b53436-50cd-44c1-b346-f7052f58c2be admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:03.320503 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b53436-50cd-44c1-b346-f7052f58c2be admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:03.321327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b53436-50cd-44c1-b346-f7052f58c2be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:03.322200 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1087/2114] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:03 2022] POST /identity/v3/users => generated 288 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:03.323582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b53436-50cd-44c1-b346-f7052f58c2be admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:03.327049 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ee802-977d-4332-a9ae-79e533e9fbf8 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:03.344408 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ee802-977d-4332-a9ae-79e533e9fbf8 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:03.351269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c9f16-55f0-4700-8a85-f5ea43c952da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:03.352255 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c9f16-55f0-4700-8a85-f5ea43c952da admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:03.352490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c9f16-55f0-4700-8a85-f5ea43c952da admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:03.352776 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c9f16-55f0-4700-8a85-f5ea43c952da admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:03.353528 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c9f16-55f0-4700-8a85-f5ea43c952da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:03.355421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c9f16-55f0-4700-8a85-f5ea43c952da admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:03.365384 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1028/2115] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:03 2022] POST /identity/v3/projects => generated 373 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:03.370579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1088/2116] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:03.371298 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b53436-50cd-44c1-b346-f7052f58c2be admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:03.376156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c9f16-55f0-4700-8a85-f5ea43c952da admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:03.391734 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c9f16-55f0-4700-8a85-f5ea43c952da admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:03.471291 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b53436-50cd-44c1-b346-f7052f58c2be admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:03.483051 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ded2c0-631c-4a01-8ae9-c70fdd2baa94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:03.483997 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ded2c0-631c-4a01-8ae9-c70fdd2baa94 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:03.484455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ded2c0-631c-4a01-8ae9-c70fdd2baa94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:03.484928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ded2c0-631c-4a01-8ae9-c70fdd2baa94 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:03.485810 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ded2c0-631c-4a01-8ae9-c70fdd2baa94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:03.487770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ded2c0-631c-4a01-8ae9-c70fdd2baa94 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:03.492492 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af0654-1324-46d2-aa10-79dce8326571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:03.493614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af0654-1324-46d2-aa10-79dce8326571 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:03.494000 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af0654-1324-46d2-aa10-79dce8326571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:03.494340 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af0654-1324-46d2-aa10-79dce8326571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:03.495432 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af0654-1324-46d2-aa10-79dce8326571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:03.499711 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af0654-1324-46d2-aa10-79dce8326571 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:03.516862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1089/2117] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:34:03.517827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af0654-1324-46d2-aa10-79dce8326571 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:03.523503 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-52af0654-1324-46d2-aa10-79dce8326571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:03.526022 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1029/2118] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:03 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:03.527095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ded2c0-631c-4a01-8ae9-c70fdd2baa94 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:03.545048 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af0654-1324-46d2-aa10-79dce8326571 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:03.545335 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ded2c0-631c-4a01-8ae9-c70fdd2baa94 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:03.552922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb237c-d9b9-4611-8627-2427fed2402c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:03.554175 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb237c-d9b9-4611-8627-2427fed2402c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:03.554496 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb237c-d9b9-4611-8627-2427fed2402c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:03.554811 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb237c-d9b9-4611-8627-2427fed2402c admin admin] PATH_INFO: `/v3/projects/bdcd47f8d1ae4b60af7f042c06d0700e` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:03.556113 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb237c-d9b9-4611-8627-2427fed2402c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdcd47f8d1ae4b60af7f042c06d0700e)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:03.559731 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb237c-d9b9-4611-8627-2427fed2402c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:03.559904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d543b-1428-4578-be04-05a46c4e2d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:03.560247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d543b-1428-4578-be04-05a46c4e2d7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:03.560532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d543b-1428-4578-be04-05a46c4e2d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:03.560806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d543b-1428-4578-be04-05a46c4e2d7e admin admin] PATH_INFO: `/v3/projects/8d1ea3d3f0ac4113bff455db1332a94f/users/45c5cb411b764414a44730891a901bdf/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:03.583769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d543b-1428-4578-be04-05a46c4e2d7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d1ea3d3f0ac4113bff455db1332a94f, user_id=45c5cb411b764414a44730891a901bdf, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:03.586556 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d543b-1428-4578-be04-05a46c4e2d7e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:03.617149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1030/2119] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:03 2022] PUT /identity/v3/projects/8d1ea3d3f0ac4113bff455db1332a94f/users/45c5cb411b764414a44730891a901bdf/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:03.617149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d543b-1428-4578-be04-05a46c4e2d7e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:03.646989 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-87cb237c-d9b9-4611-8627-2427fed2402c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdcd47f8d1ae4b60af7f042c06d0700e', 'request_id': 'req-87cb237c-d9b9-4611-8627-2427fed2402c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:03.656853 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-87cb237c-d9b9-4611-8627-2427fed2402c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdcd47f8d1ae4b60af7f042c06d0700e', 'request_id': 'req-87cb237c-d9b9-4611-8627-2427fed2402c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:03.664274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-87cb237c-d9b9-4611-8627-2427fed2402c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdcd47f8d1ae4b60af7f042c06d0700e', 'request_id': 'req-87cb237c-d9b9-4611-8627-2427fed2402c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:03.683453 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-87cb237c-d9b9-4611-8627-2427fed2402c admin admin] The token cache is being invalidate because project bdcd47f8d1ae4b60af7f042c06d0700e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:03.684236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-87cb237c-d9b9-4611-8627-2427fed2402c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:03.685620 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-87cb237c-d9b9-4611-8627-2427fed2402c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:03.688583 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1090/2120] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:03 2022] DELETE /identity/v3/projects/bdcd47f8d1ae4b60af7f042c06d0700e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:03.694885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e76c3b-0dbf-4fab-83c3-ca3a4cf39b85 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:03.695208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e76c3b-0dbf-4fab-83c3-ca3a4cf39b85 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:03.695462 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e76c3b-0dbf-4fab-83c3-ca3a4cf39b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:03.703258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:03.709183 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d543b-1428-4578-be04-05a46c4e2d7e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:03.734297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:03.735143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a8e76c3b-0dbf-4fab-83c3-ca3a4cf39b85 None None] MFA Rules not processed for user `45c5cb411b764414a44730891a901bdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:03.736778 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a8e76c3b-0dbf-4fab-83c3-ca3a4cf39b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:03.786826 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410dc23-39d8-4b48-b04a-dab0d7abbbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:03.787473 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410dc23-39d8-4b48-b04a-dab0d7abbbbc admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:03.787752 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410dc23-39d8-4b48-b04a-dab0d7abbbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:03.789000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410dc23-39d8-4b48-b04a-dab0d7abbbbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:03.789670 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0410dc23-39d8-4b48-b04a-dab0d7abbbbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:03.791355 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0410dc23-39d8-4b48-b04a-dab0d7abbbbc admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:03.803682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1031/2121] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:03.803682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410dc23-39d8-4b48-b04a-dab0d7abbbbc admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:03.803682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0410dc23-39d8-4b48-b04a-dab0d7abbbbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:03.820499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410dc23-39d8-4b48-b04a-dab0d7abbbbc admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:03.828951 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c71232-a20a-413d-8460-df37a8d45484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:03.830036 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c71232-a20a-413d-8460-df37a8d45484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:03.830423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c71232-a20a-413d-8460-df37a8d45484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:03.830781 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c71232-a20a-413d-8460-df37a8d45484 admin admin] PATH_INFO: `/v3/users/75ad613e44db40d486e78d5b4b0ea479` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:03.832445 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c71232-a20a-413d-8460-df37a8d45484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75ad613e44db40d486e78d5b4b0ea479)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:03.834856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c71232-a20a-413d-8460-df37a8d45484 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:03.853420 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1091/2122] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:03.854101 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e76c3b-0dbf-4fab-83c3-ca3a4cf39b85 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:03.856032 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a8e76c3b-0dbf-4fab-83c3-ca3a4cf39b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:03.870741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e76c3b-0dbf-4fab-83c3-ca3a4cf39b85 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:03.876416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25e86e-386f-48e6-ae68-9e651d9c3b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:03.877193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25e86e-386f-48e6-ae68-9e651d9c3b8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:03.877480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25e86e-386f-48e6-ae68-9e651d9c3b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:03.877759 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25e86e-386f-48e6-ae68-9e651d9c3b8d admin admin] PATH_INFO: `/v3/users/2d46f477264d4f2ca008278e673075e5` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:03.879185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa25e86e-386f-48e6-ae68-9e651d9c3b8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d46f477264d4f2ca008278e673075e5)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:03.881616 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa25e86e-386f-48e6-ae68-9e651d9c3b8d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:03.908240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a5c71232-a20a-413d-8460-df37a8d45484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75ad613e44db40d486e78d5b4b0ea479'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:03.919188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a5c71232-a20a-413d-8460-df37a8d45484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75ad613e44db40d486e78d5b4b0ea479'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:03.929941 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a5c71232-a20a-413d-8460-df37a8d45484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75ad613e44db40d486e78d5b4b0ea479'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:03.930867 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a5c71232-a20a-413d-8460-df37a8d45484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75ad613e44db40d486e78d5b4b0ea479'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:03.944271 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-aa25e86e-386f-48e6-ae68-9e651d9c3b8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d46f477264d4f2ca008278e673075e5'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:03.946909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-a5c71232-a20a-413d-8460-df37a8d45484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75ad613e44db40d486e78d5b4b0ea479'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:03.952196 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-aa25e86e-386f-48e6-ae68-9e651d9c3b8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d46f477264d4f2ca008278e673075e5'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:03.954137 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1032/2123] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:03 2022] DELETE /identity/v3/users/75ad613e44db40d486e78d5b4b0ea479 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:03.955134 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c71232-a20a-413d-8460-df37a8d45484 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:03.956622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a5c71232-a20a-413d-8460-df37a8d45484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:03.973727 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-aa25e86e-386f-48e6-ae68-9e651d9c3b8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d46f477264d4f2ca008278e673075e5'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:03.974723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-aa25e86e-386f-48e6-ae68-9e651d9c3b8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d46f477264d4f2ca008278e673075e5'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:03.980857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-aa25e86e-386f-48e6-ae68-9e651d9c3b8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d46f477264d4f2ca008278e673075e5'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:04.017396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1092/2124] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:03 2022] DELETE /identity/v3/users/2d46f477264d4f2ca008278e673075e5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:04.019135 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25e86e-386f-48e6-ae68-9e651d9c3b8d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:04.023313 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-aa25e86e-386f-48e6-ae68-9e651d9c3b8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:04.049706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c71232-a20a-413d-8460-df37a8d45484 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:04.095622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:04.120483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25e86e-386f-48e6-ae68-9e651d9c3b8d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:04.126644 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884aa11a-f9af-4bdb-a0bd-899ef1a5b414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:04.127847 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884aa11a-f9af-4bdb-a0bd-899ef1a5b414 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:04.128349 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884aa11a-f9af-4bdb-a0bd-899ef1a5b414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:04.129016 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884aa11a-f9af-4bdb-a0bd-899ef1a5b414 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:04.129810 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884aa11a-f9af-4bdb-a0bd-899ef1a5b414 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:04.132770 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884aa11a-f9af-4bdb-a0bd-899ef1a5b414 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:04.149286 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2feff-df96-454a-9a31-5280fa69759b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:04.149930 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2feff-df96-454a-9a31-5280fa69759b admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:04.150204 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2feff-df96-454a-9a31-5280fa69759b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:04.150483 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2feff-df96-454a-9a31-5280fa69759b admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:04.151234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c2feff-df96-454a-9a31-5280fa69759b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:04.153282 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c2feff-df96-454a-9a31-5280fa69759b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:04.168166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1033/2125] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:04.170149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1854b2-6c0b-4123-a7cf-6e414f2c8ab0 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:04.170691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1854b2-6c0b-4123-a7cf-6e414f2c8ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:04.171644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1854b2-6c0b-4123-a7cf-6e414f2c8ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:04.175590 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:04.205308 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-4d1854b2-6c0b-4123-a7cf-6e414f2c8ab0 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:04.208979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4d1854b2-6c0b-4123-a7cf-6e414f2c8ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:04.217375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1034/2126] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:04.218627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1854b2-6c0b-4123-a7cf-6e414f2c8ab0 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:04.229656 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4d1854b2-6c0b-4123-a7cf-6e414f2c8ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:04.233611 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1093/2127] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:04 2022] POST /identity/v3/projects => generated 375 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:04.234637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884aa11a-f9af-4bdb-a0bd-899ef1a5b414 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:04.244904 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-884aa11a-f9af-4bdb-a0bd-899ef1a5b414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:04.245914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1854b2-6c0b-4123-a7cf-6e414f2c8ab0 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:04.356599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09cf64-b702-4311-8015-f4bf7f999b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:04.357735 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09cf64-b702-4311-8015-f4bf7f999b76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:04.358148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09cf64-b702-4311-8015-f4bf7f999b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:04.358692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09cf64-b702-4311-8015-f4bf7f999b76 admin admin] PATH_INFO: `/v3/projects/973b4a0dd723403b884f5a1050e85a4e/users/12ee7183e3064ca184156234296594fe/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:04.389255 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884aa11a-f9af-4bdb-a0bd-899ef1a5b414 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:04.393566 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907d129-ef69-4865-88ba-89332a60b10c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:04.394607 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907d129-ef69-4865-88ba-89332a60b10c service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:04.395054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907d129-ef69-4865-88ba-89332a60b10c service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:04.396285 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907d129-ef69-4865-88ba-89332a60b10c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:04.404089 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09cf64-b702-4311-8015-f4bf7f999b76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=973b4a0dd723403b884f5a1050e85a4e, user_id=12ee7183e3064ca184156234296594fe, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:04.408474 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907d129-ef69-4865-88ba-89332a60b10c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:04.410015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907d129-ef69-4865-88ba-89332a60b10c service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:04.412900 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09cf64-b702-4311-8015-f4bf7f999b76 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:04.431538 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1094/2128] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:04.433024 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907d129-ef69-4865-88ba-89332a60b10c service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:04.459997 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907d129-ef69-4865-88ba-89332a60b10c service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:04.460353 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1035/2129] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:04 2022] PUT /identity/v3/projects/973b4a0dd723403b884f5a1050e85a4e/users/12ee7183e3064ca184156234296594fe/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:04.467536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09cf64-b702-4311-8015-f4bf7f999b76 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:04.478327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1e09cf64-b702-4311-8015-f4bf7f999b76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:04.578269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28490cf5-1bc8-405b-8aaa-e050ad046c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:04.579405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28490cf5-1bc8-405b-8aaa-e050ad046c77 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:04.579892 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28490cf5-1bc8-405b-8aaa-e050ad046c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:04.580576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28490cf5-1bc8-405b-8aaa-e050ad046c77 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:04.581700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28490cf5-1bc8-405b-8aaa-e050ad046c77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:04.584897 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28490cf5-1bc8-405b-8aaa-e050ad046c77 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:04.587097 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09cf64-b702-4311-8015-f4bf7f999b76 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:04.593011 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b7cd9-ca6e-4907-934a-216477ad9b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:04.593657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b7cd9-ca6e-4907-934a-216477ad9b43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:04.593935 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b7cd9-ca6e-4907-934a-216477ad9b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:04.594204 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b7cd9-ca6e-4907-934a-216477ad9b43 admin admin] PATH_INFO: `/v3/projects/7d35942f930c4ed79efd56176eec81b4` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:04.595344 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b7cd9-ca6e-4907-934a-216477ad9b43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d35942f930c4ed79efd56176eec81b4)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:04.597954 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b7cd9-ca6e-4907-934a-216477ad9b43 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:04.615236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1095/2130] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:04 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:04.618665 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c05a7-0eb4-4194-8df9-c089752ddc00 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:04.619427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c05a7-0eb4-4194-8df9-c089752ddc00 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:04.620085 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c05a7-0eb4-4194-8df9-c089752ddc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:04.660023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-250b7cd9-ca6e-4907-934a-216477ad9b43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d35942f930c4ed79efd56176eec81b4', 'request_id': 'req-250b7cd9-ca6e-4907-934a-216477ad9b43'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:04.660130 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-521c05a7-0eb4-4194-8df9-c089752ddc00 None None] MFA Rules not processed for user `12ee7183e3064ca184156234296594fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:04.661907 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-521c05a7-0eb4-4194-8df9-c089752ddc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:04.663501 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:04.670112 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-250b7cd9-ca6e-4907-934a-216477ad9b43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d35942f930c4ed79efd56176eec81b4', 'request_id': 'req-250b7cd9-ca6e-4907-934a-216477ad9b43'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:04.680276 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-250b7cd9-ca6e-4907-934a-216477ad9b43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d35942f930c4ed79efd56176eec81b4', 'request_id': 'req-250b7cd9-ca6e-4907-934a-216477ad9b43'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:04.695520 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-250b7cd9-ca6e-4907-934a-216477ad9b43 admin admin] The token cache is being invalidate because project 7d35942f930c4ed79efd56176eec81b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:04.695851 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-250b7cd9-ca6e-4907-934a-216477ad9b43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:04.696677 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-250b7cd9-ca6e-4907-934a-216477ad9b43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:04.699926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1036/2131] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:04 2022] DELETE /identity/v3/projects/7d35942f930c4ed79efd56176eec81b4 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:34:04.700972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b7cd9-ca6e-4907-934a-216477ad9b43 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:04.702421 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-250b7cd9-ca6e-4907-934a-216477ad9b43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:04.756886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1096/2132] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:04.758197 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df615be3-0db2-4ed1-a81c-aa11920203f9 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:04.758487 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df615be3-0db2-4ed1-a81c-aa11920203f9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:04.758762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df615be3-0db2-4ed1-a81c-aa11920203f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:04.781475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-df615be3-0db2-4ed1-a81c-aa11920203f9 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:04.783243 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-df615be3-0db2-4ed1-a81c-aa11920203f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:04.788708 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b7cd9-ca6e-4907-934a-216477ad9b43 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:04.795398 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e106a197-3220-4298-a228-c07188f47f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:04.796104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106a197-3220-4298-a228-c07188f47f0a admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:04.796260 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1097/2133] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:04.800142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106a197-3220-4298-a228-c07188f47f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:04.800142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e106a197-3220-4298-a228-c07188f47f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:04.800482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df615be3-0db2-4ed1-a81c-aa11920203f9 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:04.800601 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e106a197-3220-4298-a228-c07188f47f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:04.800601 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e106a197-3220-4298-a228-c07188f47f0a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:04.805747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-df615be3-0db2-4ed1-a81c-aa11920203f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:04.820002 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1037/2134] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:04.820002 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332a2e3-a9fe-4c2b-9b36-c0371753dc56 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:04.820002 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332a2e3-a9fe-4c2b-9b36-c0371753dc56 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:04.820002 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6332a2e3-a9fe-4c2b-9b36-c0371753dc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:04.820992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:04.844666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6332a2e3-a9fe-4c2b-9b36-c0371753dc56 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:04.848094 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6332a2e3-a9fe-4c2b-9b36-c0371753dc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:04.859141 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1038/2135] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:04.863080 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6332a2e3-a9fe-4c2b-9b36-c0371753dc56 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:04.863080 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6332a2e3-a9fe-4c2b-9b36-c0371753dc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:04.874967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6332a2e3-a9fe-4c2b-9b36-c0371753dc56 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:04.880170 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eab87e-33e4-46ae-8d65-21a2abaf5643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:04.880819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eab87e-33e4-46ae-8d65-21a2abaf5643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:04.881100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eab87e-33e4-46ae-8d65-21a2abaf5643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:04.881368 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eab87e-33e4-46ae-8d65-21a2abaf5643 admin admin] PATH_INFO: `/v3/projects/deb33d138b0f4965af6f62b28c534083` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:04.882574 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eab87e-33e4-46ae-8d65-21a2abaf5643 admin admin] RBAC: Authorizing `identity:delete_project(project_id=deb33d138b0f4965af6f62b28c534083)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:04.884693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eab87e-33e4-46ae-8d65-21a2abaf5643 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:04.888968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df615be3-0db2-4ed1-a81c-aa11920203f9 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:04.892780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc74d9-be2b-4b99-ae94-7398c0c34d78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:04.893405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc74d9-be2b-4b99-ae94-7398c0c34d78 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:04.893688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc74d9-be2b-4b99-ae94-7398c0c34d78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:04.894041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc74d9-be2b-4b99-ae94-7398c0c34d78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:04.904497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc74d9-be2b-4b99-ae94-7398c0c34d78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:04.906139 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc74d9-be2b-4b99-ae94-7398c0c34d78 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:04.916640 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1098/2136] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:04.918269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc74d9-be2b-4b99-ae94-7398c0c34d78 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:04.931395 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc74d9-be2b-4b99-ae94-7398c0c34d78 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:04.940685 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5bcf6-ac13-4eeb-8d8f-12b17ce90c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:04.941905 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5bcf6-ac13-4eeb-8d8f-12b17ce90c56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:04.942656 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5bcf6-ac13-4eeb-8d8f-12b17ce90c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:04.943217 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5bcf6-ac13-4eeb-8d8f-12b17ce90c56 admin admin] PATH_INFO: `/v3/projects/55f63880897a4d2381bc0ee4ae6e2113/users/da281ba9a13b4287a143ed65a642aadb/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:04.960729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-61eab87e-33e4-46ae-8d65-21a2abaf5643 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deb33d138b0f4965af6f62b28c534083', 'request_id': 'req-61eab87e-33e4-46ae-8d65-21a2abaf5643'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:04.967298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5bcf6-ac13-4eeb-8d8f-12b17ce90c56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55f63880897a4d2381bc0ee4ae6e2113, user_id=da281ba9a13b4287a143ed65a642aadb, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:04.968905 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-61eab87e-33e4-46ae-8d65-21a2abaf5643 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deb33d138b0f4965af6f62b28c534083', 'request_id': 'req-61eab87e-33e4-46ae-8d65-21a2abaf5643'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:04.970177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5bcf6-ac13-4eeb-8d8f-12b17ce90c56 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:04.981398 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-61eab87e-33e4-46ae-8d65-21a2abaf5643 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deb33d138b0f4965af6f62b28c534083', 'request_id': 'req-61eab87e-33e4-46ae-8d65-21a2abaf5643'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:04.988788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1099/2137] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:04 2022] PUT /identity/v3/projects/55f63880897a4d2381bc0ee4ae6e2113/users/da281ba9a13b4287a143ed65a642aadb/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:04.992783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5bcf6-ac13-4eeb-8d8f-12b17ce90c56 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:04.996847 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-61eab87e-33e4-46ae-8d65-21a2abaf5643 admin admin] The token cache is being invalidate because project deb33d138b0f4965af6f62b28c534083 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:04.997787 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-61eab87e-33e4-46ae-8d65-21a2abaf5643 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:04.999136 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-61eab87e-33e4-46ae-8d65-21a2abaf5643 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:05.001660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1039/2138] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:04 2022] DELETE /identity/v3/projects/deb33d138b0f4965af6f62b28c534083 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:05.003415 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eab87e-33e4-46ae-8d65-21a2abaf5643 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:05.007320 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-61eab87e-33e4-46ae-8d65-21a2abaf5643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:05.069823 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:05.074940 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5bcf6-ac13-4eeb-8d8f-12b17ce90c56 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:05.081024 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe35f8d-2f1f-46c8-a886-b20ed8b3eb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:05.081643 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe35f8d-2f1f-46c8-a886-b20ed8b3eb91 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:05.081926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe35f8d-2f1f-46c8-a886-b20ed8b3eb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:05.082200 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe35f8d-2f1f-46c8-a886-b20ed8b3eb91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:05.082867 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe35f8d-2f1f-46c8-a886-b20ed8b3eb91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:34:05.083565 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe35f8d-2f1f-46c8-a886-b20ed8b3eb91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:05.085809 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe35f8d-2f1f-46c8-a886-b20ed8b3eb91 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:05.090019 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eab87e-33e4-46ae-8d65-21a2abaf5643 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:05.097766 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c417bd-d547-4ef6-baa5-bdf7afa28944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:05.098745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c417bd-d547-4ef6-baa5-bdf7afa28944 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:05.098745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c417bd-d547-4ef6-baa5-bdf7afa28944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:05.099044 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c417bd-d547-4ef6-baa5-bdf7afa28944 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:05.099967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c417bd-d547-4ef6-baa5-bdf7afa28944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:05.100157 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1100/2139] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:34:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:34:05.102268 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c417bd-d547-4ef6-baa5-bdf7afa28944 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:05.105477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe35f8d-2f1f-46c8-a886-b20ed8b3eb91 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:05.108345 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-bbe35f8d-2f1f-46c8-a886-b20ed8b3eb91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:05.113745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1040/2140] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:05.114822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c417bd-d547-4ef6-baa5-bdf7afa28944 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:05.118698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-47c417bd-d547-4ef6-baa5-bdf7afa28944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:05.124391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe35f8d-2f1f-46c8-a886-b20ed8b3eb91 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:05.130270 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dacc9fb-7f5e-4505-86f9-df82995d4a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:05.130984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dacc9fb-7f5e-4505-86f9-df82995d4a30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:05.131823 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dacc9fb-7f5e-4505-86f9-df82995d4a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:05.131973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dacc9fb-7f5e-4505-86f9-df82995d4a30 admin admin] PATH_INFO: `/v3/users/7b2277c33fa740f4ba98f44258849ad1` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:05.133122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dacc9fb-7f5e-4505-86f9-df82995d4a30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b2277c33fa740f4ba98f44258849ad1)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:05.134495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c417bd-d547-4ef6-baa5-bdf7afa28944 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:05.135099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dacc9fb-7f5e-4505-86f9-df82995d4a30 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:05.141622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e2617-a515-4fe6-8565-d36b880e37a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:05.141622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e2617-a515-4fe6-8565-d36b880e37a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:05.142007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e2617-a515-4fe6-8565-d36b880e37a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:05.142280 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e2617-a515-4fe6-8565-d36b880e37a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:05.142905 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e2617-a515-4fe6-8565-d36b880e37a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:05.144865 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e2617-a515-4fe6-8565-d36b880e37a7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:05.182260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1041/2141] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:05 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:05.183399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e2617-a515-4fe6-8565-d36b880e37a7 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:05.209066 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3dacc9fb-7f5e-4505-86f9-df82995d4a30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b2277c33fa740f4ba98f44258849ad1'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:05.219770 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3dacc9fb-7f5e-4505-86f9-df82995d4a30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b2277c33fa740f4ba98f44258849ad1'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:05.235512 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3dacc9fb-7f5e-4505-86f9-df82995d4a30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b2277c33fa740f4ba98f44258849ad1'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:05.238366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3dacc9fb-7f5e-4505-86f9-df82995d4a30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b2277c33fa740f4ba98f44258849ad1'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:05.241072 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3dacc9fb-7f5e-4505-86f9-df82995d4a30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b2277c33fa740f4ba98f44258849ad1'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:05.253325 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:05.278399 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1101/2142] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:05 2022] DELETE /identity/v3/users/7b2277c33fa740f4ba98f44258849ad1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:05.280298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dacc9fb-7f5e-4505-86f9-df82995d4a30 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:05.284288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3dacc9fb-7f5e-4505-86f9-df82995d4a30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:05.357905 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e2617-a515-4fe6-8565-d36b880e37a7 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:05.372961 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7c7ad-f92d-4b89-955f-3b3feb07962b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:05.375769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7c7ad-f92d-4b89-955f-3b3feb07962b admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:05.375980 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dacc9fb-7f5e-4505-86f9-df82995d4a30 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:05.376317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7c7ad-f92d-4b89-955f-3b3feb07962b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:05.377994 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7c7ad-f92d-4b89-955f-3b3feb07962b admin admin] PATH_INFO: `/v3/projects/55f63880897a4d2381bc0ee4ae6e2113/users/da281ba9a13b4287a143ed65a642aadb/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:05.383430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7c7ad-f92d-4b89-955f-3b3feb07962b admin admin] RBAC: Authorizing `identity:create_grant(project_id=55f63880897a4d2381bc0ee4ae6e2113, user_id=da281ba9a13b4287a143ed65a642aadb, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:05.383818 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06c47e-a910-4b18-8a9d-8d2b92424b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:05.384638 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06c47e-a910-4b18-8a9d-8d2b92424b80 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:05.385139 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06c47e-a910-4b18-8a9d-8d2b92424b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:05.385557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06c47e-a910-4b18-8a9d-8d2b92424b80 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:05.386511 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06c47e-a910-4b18-8a9d-8d2b92424b80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:05.391914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06c47e-a910-4b18-8a9d-8d2b92424b80 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:05.392511 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7c7ad-f92d-4b89-955f-3b3feb07962b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:05.435546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1102/2143] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:05 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:05.440176 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06c47e-a910-4b18-8a9d-8d2b92424b80 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:05.440739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1042/2144] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:05 2022] PUT /identity/v3/projects/55f63880897a4d2381bc0ee4ae6e2113/users/da281ba9a13b4287a143ed65a642aadb/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:05.452373 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44bc42-e188-442d-ab4e-75c3fad8e231 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:05.452681 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44bc42-e188-442d-ab4e-75c3fad8e231 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:05.452944 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44bc42-e188-442d-ab4e-75c3fad8e231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:05.463685 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06c47e-a910-4b18-8a9d-8d2b92424b80 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:05.504666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ae44bc42-e188-442d-ab4e-75c3fad8e231 None None] MFA Rules not processed for user `da281ba9a13b4287a143ed65a642aadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:05.506702 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ae44bc42-e188-442d-ab4e-75c3fad8e231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:05.552207 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09541887-afd0-4ea8-9828-5128611fa206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:05.556284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09541887-afd0-4ea8-9828-5128611fa206 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:05.556844 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09541887-afd0-4ea8-9828-5128611fa206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:05.557443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09541887-afd0-4ea8-9828-5128611fa206 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:05.558318 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09541887-afd0-4ea8-9828-5128611fa206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:05.564768 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09541887-afd0-4ea8-9828-5128611fa206 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:05.590165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1103/2145] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:05.592902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09541887-afd0-4ea8-9828-5128611fa206 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:05.609564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09541887-afd0-4ea8-9828-5128611fa206 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:05.614700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f957390-d9d7-4cee-a844-f93bf5aa2ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:05.616456 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f957390-d9d7-4cee-a844-f93bf5aa2ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:05.617140 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f957390-d9d7-4cee-a844-f93bf5aa2ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:05.617434 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f957390-d9d7-4cee-a844-f93bf5aa2ac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:05.618801 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f957390-d9d7-4cee-a844-f93bf5aa2ac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:05.620885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f957390-d9d7-4cee-a844-f93bf5aa2ac0 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:05.640011 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1104/2146] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:05.640500 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1043/2147] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:34:05.646412 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f957390-d9d7-4cee-a844-f93bf5aa2ac0 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:05.662125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44bc42-e188-442d-ab4e-75c3fad8e231 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:05.664188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ae44bc42-e188-442d-ab4e-75c3fad8e231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:05.672625 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f957390-d9d7-4cee-a844-f93bf5aa2ac0 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:05.692141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237832d5-bf26-4511-98b8-b7129e6245b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:05.692939 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237832d5-bf26-4511-98b8-b7129e6245b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:05.694150 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237832d5-bf26-4511-98b8-b7129e6245b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:05.696102 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237832d5-bf26-4511-98b8-b7129e6245b7 admin admin] PATH_INFO: `/v3/projects/93e26c92435045df8b4e76a9f9403cc5/users/076da899d1084121b8d96c8c9da965ce/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:05.720804 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237832d5-bf26-4511-98b8-b7129e6245b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93e26c92435045df8b4e76a9f9403cc5, user_id=076da899d1084121b8d96c8c9da965ce, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:05.724781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237832d5-bf26-4511-98b8-b7129e6245b7 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:05.748879 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1105/2148] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:05 2022] PUT /identity/v3/projects/93e26c92435045df8b4e76a9f9403cc5/users/076da899d1084121b8d96c8c9da965ce/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:05.753998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237832d5-bf26-4511-98b8-b7129e6245b7 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:05.760271 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-237832d5-bf26-4511-98b8-b7129e6245b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:05.762514 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:05.781991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44bc42-e188-442d-ab4e-75c3fad8e231 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:05.845648 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:05.858003 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3687ee-78a9-40ef-a7da-77b2b482c429 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:05.858626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3687ee-78a9-40ef-a7da-77b2b482c429 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:05.858985 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3687ee-78a9-40ef-a7da-77b2b482c429 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:05.859222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3687ee-78a9-40ef-a7da-77b2b482c429 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:05.861652 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5b3687ee-78a9-40ef-a7da-77b2b482c429 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:05.873144 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237832d5-bf26-4511-98b8-b7129e6245b7 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:05.879041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75078a-1795-4622-abe2-333e772e9379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:05.880378 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75078a-1795-4622-abe2-333e772e9379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:05.880663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75078a-1795-4622-abe2-333e772e9379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:05.880943 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75078a-1795-4622-abe2-333e772e9379 admin admin] PATH_INFO: `/v3/projects/fc1ccdeeb0274b89a0e7c51f317a3c9f` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:05.882678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e75078a-1795-4622-abe2-333e772e9379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc1ccdeeb0274b89a0e7c51f317a3c9f)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:05.884622 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e75078a-1795-4622-abe2-333e772e9379 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:05.932452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3687ee-78a9-40ef-a7da-77b2b482c429 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:05.933849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3687ee-78a9-40ef-a7da-77b2b482c429 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:05.938572 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4e75078a-1795-4622-abe2-333e772e9379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc1ccdeeb0274b89a0e7c51f317a3c9f', 'request_id': 'req-4e75078a-1795-4622-abe2-333e772e9379'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:05.943438 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1044/2149] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:34:05.945549 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7da003-aad6-4666-b78e-634cd51c184d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:05.945890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7da003-aad6-4666-b78e-634cd51c184d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:05.946249 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7da003-aad6-4666-b78e-634cd51c184d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:05.947722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4e75078a-1795-4622-abe2-333e772e9379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc1ccdeeb0274b89a0e7c51f317a3c9f', 'request_id': 'req-4e75078a-1795-4622-abe2-333e772e9379'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:05.954321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4e75078a-1795-4622-abe2-333e772e9379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc1ccdeeb0274b89a0e7c51f317a3c9f', 'request_id': 'req-4e75078a-1795-4622-abe2-333e772e9379'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:05.968745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4e75078a-1795-4622-abe2-333e772e9379 admin admin] The token cache is being invalidate because project fc1ccdeeb0274b89a0e7c51f317a3c9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:05.969041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4e75078a-1795-4622-abe2-333e772e9379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:05.969831 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4e75078a-1795-4622-abe2-333e772e9379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:05.972737 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1106/2150] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:05 2022] DELETE /identity/v3/projects/fc1ccdeeb0274b89a0e7c51f317a3c9f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:34:05.978042 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25439502-e58b-4f3a-a9e1-b2b5ec7cad74 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:05.978468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25439502-e58b-4f3a-a9e1-b2b5ec7cad74 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:05.978756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25439502-e58b-4f3a-a9e1-b2b5ec7cad74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:05.980005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5e7da003-aad6-4666-b78e-634cd51c184d None None] MFA Rules not processed for user `076da899d1084121b8d96c8c9da965ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:05.982384 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5e7da003-aad6-4666-b78e-634cd51c184d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:06.004554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-25439502-e58b-4f3a-a9e1-b2b5ec7cad74 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:06.005829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-25439502-e58b-4f3a-a9e1-b2b5ec7cad74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:06.090645 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1045/2151] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:06.094192 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7da003-aad6-4666-b78e-634cd51c184d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:06.097198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5e7da003-aad6-4666-b78e-634cd51c184d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:06.123373 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1107/2152] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:34:06.131151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67745a-283a-421b-b6b1-d172d44cd4fd None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:06.131505 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67745a-283a-421b-b6b1-d172d44cd4fd None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:06.132335 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67745a-283a-421b-b6b1-d172d44cd4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:06.158663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-de67745a-283a-421b-b6b1-d172d44cd4fd None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:06.159911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-de67745a-283a-421b-b6b1-d172d44cd4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:06.174962 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1108/2153] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:06.180832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67745a-283a-421b-b6b1-d172d44cd4fd None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:06.186713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:06.194817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67745a-283a-421b-b6b1-d172d44cd4fd None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:06.200921 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f3af5-e729-4ff6-9007-46b45f8fb71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:06.201557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f3af5-e729-4ff6-9007-46b45f8fb71d admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:06.201840 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f3af5-e729-4ff6-9007-46b45f8fb71d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:06.202153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f3af5-e729-4ff6-9007-46b45f8fb71d admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:06.202964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f3af5-e729-4ff6-9007-46b45f8fb71d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:34:06.203274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f3af5-e729-4ff6-9007-46b45f8fb71d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:06.206057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f3af5-e729-4ff6-9007-46b45f8fb71d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:06.211843 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7da003-aad6-4666-b78e-634cd51c184d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:06.219167 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1109/2154] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:34:06.223128 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13eec5-2a0e-435f-92e8-4c60dfce5f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:06.223128 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13eec5-2a0e-435f-92e8-4c60dfce5f12 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:06.223128 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13eec5-2a0e-435f-92e8-4c60dfce5f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:06.223128 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13eec5-2a0e-435f-92e8-4c60dfce5f12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:06.223483 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13eec5-2a0e-435f-92e8-4c60dfce5f12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:06.224145 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f3af5-e729-4ff6-9007-46b45f8fb71d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:06.224789 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13eec5-2a0e-435f-92e8-4c60dfce5f12 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:06.235756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f3af5-e729-4ff6-9007-46b45f8fb71d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:06.242441 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4586c02-b052-4393-9b1a-a41048743d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:06.243225 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4586c02-b052-4393-9b1a-a41048743d26 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:06.243514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4586c02-b052-4393-9b1a-a41048743d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:06.243848 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4586c02-b052-4393-9b1a-a41048743d26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:06.244836 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4586c02-b052-4393-9b1a-a41048743d26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:06.247157 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4586c02-b052-4393-9b1a-a41048743d26 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:06.259969 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1046/2155] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:06 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:06.264038 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13eec5-2a0e-435f-92e8-4c60dfce5f12 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:06.282503 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1110/2156] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:06 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:34:06.283632 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4586c02-b052-4393-9b1a-a41048743d26 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:06.340806 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:06.362833 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13eec5-2a0e-435f-92e8-4c60dfce5f12 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:06.407250 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4586c02-b052-4393-9b1a-a41048743d26 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:06.414628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa3bd3-18e9-4aaf-8174-d218ad8dcc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:06.415384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa3bd3-18e9-4aaf-8174-d218ad8dcc1e admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:06.416054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa3bd3-18e9-4aaf-8174-d218ad8dcc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:06.416569 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa3bd3-18e9-4aaf-8174-d218ad8dcc1e admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:06.417761 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa3bd3-18e9-4aaf-8174-d218ad8dcc1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:06.424209 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa3bd3-18e9-4aaf-8174-d218ad8dcc1e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:06.463209 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1111/2157] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:06 2022] POST /identity/v3/users => generated 291 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:34:06.464938 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa3bd3-18e9-4aaf-8174-d218ad8dcc1e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:06.470667 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf41d90-d73d-4f92-b35c-4491e562a7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:06.472782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf41d90-d73d-4f92-b35c-4491e562a7e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:06.473151 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf41d90-d73d-4f92-b35c-4491e562a7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:06.473612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf41d90-d73d-4f92-b35c-4491e562a7e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:06.474296 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf41d90-d73d-4f92-b35c-4491e562a7e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:06.477323 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf41d90-d73d-4f92-b35c-4491e562a7e4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:06.480591 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa3bd3-18e9-4aaf-8174-d218ad8dcc1e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:06.502904 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ac0c8-32e3-4257-b243-8e5faba65913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:06.504458 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ac0c8-32e3-4257-b243-8e5faba65913 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:06.504808 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ac0c8-32e3-4257-b243-8e5faba65913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:06.506297 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ac0c8-32e3-4257-b243-8e5faba65913 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:06.508234 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ac0c8-32e3-4257-b243-8e5faba65913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:06.512445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ac0c8-32e3-4257-b243-8e5faba65913 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:06.534332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1047/2158] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:06 2022] POST /identity/v3/users => generated 290 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:34:06.536469 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf41d90-d73d-4f92-b35c-4491e562a7e4 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:06.538863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1112/2159] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:06.543405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ac0c8-32e3-4257-b243-8e5faba65913 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:06.559533 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ac0c8-32e3-4257-b243-8e5faba65913 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:06.562216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf41d90-d73d-4f92-b35c-4491e562a7e4 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:06.565808 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3679ff0-51fe-4cf4-bc74-bfce8283a2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:06.566758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3679ff0-51fe-4cf4-bc74-bfce8283a2bd admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:06.567021 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3679ff0-51fe-4cf4-bc74-bfce8283a2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:06.567284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3679ff0-51fe-4cf4-bc74-bfce8283a2bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:06.568013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3679ff0-51fe-4cf4-bc74-bfce8283a2bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:06.569897 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3679ff0-51fe-4cf4-bc74-bfce8283a2bd admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:06.575071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f836bfa-bdf9-430c-9ed2-b3c24e3469fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:06.575898 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f836bfa-bdf9-430c-9ed2-b3c24e3469fe admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:06.576327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f836bfa-bdf9-430c-9ed2-b3c24e3469fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:06.576696 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f836bfa-bdf9-430c-9ed2-b3c24e3469fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:06.577540 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f836bfa-bdf9-430c-9ed2-b3c24e3469fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:06.579364 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f836bfa-bdf9-430c-9ed2-b3c24e3469fe admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:06.597453 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1113/2160] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:06.597940 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3679ff0-51fe-4cf4-bc74-bfce8283a2bd admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:06.607479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1048/2161] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:06.615099 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f836bfa-bdf9-430c-9ed2-b3c24e3469fe admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:06.615979 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3679ff0-51fe-4cf4-bc74-bfce8283a2bd admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:06.622148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a74f44-b783-43e7-950d-0fff74bfece1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:06.622834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a74f44-b783-43e7-950d-0fff74bfece1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:06.623120 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a74f44-b783-43e7-950d-0fff74bfece1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:06.623413 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a74f44-b783-43e7-950d-0fff74bfece1 admin admin] PATH_INFO: `/v3/projects/7c69548e9cee495382b398a072732fd9/users/19c00cca09944f70914d73eb51ee879a/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:06.652037 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f836bfa-bdf9-430c-9ed2-b3c24e3469fe admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:06.658425 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4a6d9-b5fe-4cb0-9e53-233b17edf836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:06.659568 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4a6d9-b5fe-4cb0-9e53-233b17edf836 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:06.660261 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4a6d9-b5fe-4cb0-9e53-233b17edf836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:06.660748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4a6d9-b5fe-4cb0-9e53-233b17edf836 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:06.662081 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4a6d9-b5fe-4cb0-9e53-233b17edf836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:06.666125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4a6d9-b5fe-4cb0-9e53-233b17edf836 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:06.674780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a74f44-b783-43e7-950d-0fff74bfece1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c69548e9cee495382b398a072732fd9, user_id=19c00cca09944f70914d73eb51ee879a, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:06.681866 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a74f44-b783-43e7-950d-0fff74bfece1 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:06.688752 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1049/2162] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:06.695871 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4a6d9-b5fe-4cb0-9e53-233b17edf836 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:06.713247 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1114/2163] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:06 2022] PUT /identity/v3/projects/7c69548e9cee495382b398a072732fd9/users/19c00cca09944f70914d73eb51ee879a/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:06.719114 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4a6d9-b5fe-4cb0-9e53-233b17edf836 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:06.731355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e1ef1-0c39-4ca1-83ae-bc97f68bf663 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:06.732008 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e1ef1-0c39-4ca1-83ae-bc97f68bf663 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:06.732450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e1ef1-0c39-4ca1-83ae-bc97f68bf663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:06.797706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-9c9e1ef1-0c39-4ca1-83ae-bc97f68bf663 None None] MFA Rules not processed for user `19c00cca09944f70914d73eb51ee879a`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:06.800183 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9c9e1ef1-0c39-4ca1-83ae-bc97f68bf663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:06.902530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:06.949973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715356e-69a9-4678-aa68-ac5075bdc13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:06.952267 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715356e-69a9-4678-aa68-ac5075bdc13e admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:06.954231 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715356e-69a9-4678-aa68-ac5075bdc13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:06.955291 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715356e-69a9-4678-aa68-ac5075bdc13e admin admin] PATH_INFO: `/v3/projects/b6b2b1737ee24d6f9ef9aa90711ff819/users/0afc65be55ba48408c556fb2b1e40476/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:06.997856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715356e-69a9-4678-aa68-ac5075bdc13e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6b2b1737ee24d6f9ef9aa90711ff819, user_id=0afc65be55ba48408c556fb2b1e40476, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:07.001882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715356e-69a9-4678-aa68-ac5075bdc13e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:07.018196 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1115/2164] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:07.032767 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e1ef1-0c39-4ca1-83ae-bc97f68bf663 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:07.049056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e1ef1-0c39-4ca1-83ae-bc97f68bf663 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:07.056196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1050/2165] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:06 2022] PUT /identity/v3/projects/b6b2b1737ee24d6f9ef9aa90711ff819/users/0afc65be55ba48408c556fb2b1e40476/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:07.073732 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c3730-2672-431c-9d8f-63e36e76fc4e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:07.074098 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c3730-2672-431c-9d8f-63e36e76fc4e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:07.074302 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c3730-2672-431c-9d8f-63e36e76fc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:07.114528 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-593c3730-2672-431c-9d8f-63e36e76fc4e None None] MFA Rules not processed for user `0afc65be55ba48408c556fb2b1e40476`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:07.115775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-593c3730-2672-431c-9d8f-63e36e76fc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:07.215809 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aade36-8938-4746-abc0-5c03d7d08e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:07.216557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aade36-8938-4746-abc0-5c03d7d08e20 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:07.217328 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aade36-8938-4746-abc0-5c03d7d08e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:07.217713 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aade36-8938-4746-abc0-5c03d7d08e20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:07.218567 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aade36-8938-4746-abc0-5c03d7d08e20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:07.221002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aade36-8938-4746-abc0-5c03d7d08e20 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:07.228177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:07.263593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1051/2166] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:07.271383 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c3730-2672-431c-9d8f-63e36e76fc4e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:07.283046 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1116/2167] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:07 2022] POST /identity/v3/projects => generated 381 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:07.285576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aade36-8938-4746-abc0-5c03d7d08e20 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:07.288888 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c3730-2672-431c-9d8f-63e36e76fc4e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:07.373867 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72ff45-2366-4833-9c51-2fca7e49e1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:07.376616 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72ff45-2366-4833-9c51-2fca7e49e1d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:07.376616 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72ff45-2366-4833-9c51-2fca7e49e1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:07.376616 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72ff45-2366-4833-9c51-2fca7e49e1d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:07.376961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aade36-8938-4746-abc0-5c03d7d08e20 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:07.377096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72ff45-2366-4833-9c51-2fca7e49e1d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:07.379517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72ff45-2366-4833-9c51-2fca7e49e1d8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:07.383769 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d8711-c9b4-4a51-95ef-e3f5d083d561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:07.384811 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d8711-c9b4-4a51-95ef-e3f5d083d561 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:07.385232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d8711-c9b4-4a51-95ef-e3f5d083d561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:07.385653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d8711-c9b4-4a51-95ef-e3f5d083d561 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:07.386638 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d8711-c9b4-4a51-95ef-e3f5d083d561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:07.389800 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d8711-c9b4-4a51-95ef-e3f5d083d561 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:07.422483 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1052/2168] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:07 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:34:07.424118 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1117/2169] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:07 2022] POST /identity/v3/users => generated 297 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:34:07.426624 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72ff45-2366-4833-9c51-2fca7e49e1d8 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:07.428581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d8711-c9b4-4a51-95ef-e3f5d083d561 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:07.443813 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d8711-c9b4-4a51-95ef-e3f5d083d561 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:07.485050 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:07.509208 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72ff45-2366-4833-9c51-2fca7e49e1d8 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:07.513130 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0213e42-4b75-4585-9c9f-00384bb428b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:07.513945 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0213e42-4b75-4585-9c9f-00384bb428b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:07.514335 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0213e42-4b75-4585-9c9f-00384bb428b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:07.514804 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0213e42-4b75-4585-9c9f-00384bb428b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:07.515959 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0213e42-4b75-4585-9c9f-00384bb428b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:07.516214 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e4fda-2997-4573-9c88-6351d2f8c412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:07.517021 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e4fda-2997-4573-9c88-6351d2f8c412 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:07.517484 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e4fda-2997-4573-9c88-6351d2f8c412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:07.517946 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e4fda-2997-4573-9c88-6351d2f8c412 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:07.518773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e4fda-2997-4573-9c88-6351d2f8c412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:07.518930 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0213e42-4b75-4585-9c9f-00384bb428b3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:07.521815 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e4fda-2997-4573-9c88-6351d2f8c412 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:07.534061 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1118/2170] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:07.537850 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0213e42-4b75-4585-9c9f-00384bb428b3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:07.551053 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0213e42-4b75-4585-9c9f-00384bb428b3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:07.555966 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1053/2171] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:07 2022] POST /identity/v3/users => generated 295 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:34:07.560478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e4fda-2997-4573-9c88-6351d2f8c412 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:07.561616 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c6698-8510-4a7d-810e-01758a9c7dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:07.562340 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c6698-8510-4a7d-810e-01758a9c7dc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:07.562653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c6698-8510-4a7d-810e-01758a9c7dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:07.562877 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c6698-8510-4a7d-810e-01758a9c7dc6 admin admin] PATH_INFO: `/v3/projects/6940d4d91e2246848bb4a46716aa8e38/users/22766b0421c645cc96078eeff2cda6b9/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:07.578628 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e4fda-2997-4573-9c88-6351d2f8c412 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:07.585590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d5c9c-0eff-4df3-93c8-31cfc66b8a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:07.586212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d5c9c-0eff-4df3-93c8-31cfc66b8a1f admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:07.586368 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c6698-8510-4a7d-810e-01758a9c7dc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6940d4d91e2246848bb4a46716aa8e38, user_id=22766b0421c645cc96078eeff2cda6b9, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:07.586517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d5c9c-0eff-4df3-93c8-31cfc66b8a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:07.586760 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d5c9c-0eff-4df3-93c8-31cfc66b8a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:07.587413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d5c9c-0eff-4df3-93c8-31cfc66b8a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:07.589171 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d5c9c-0eff-4df3-93c8-31cfc66b8a1f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:07.589534 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c6698-8510-4a7d-810e-01758a9c7dc6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:07.607522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1054/2172] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:07.609049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d5c9c-0eff-4df3-93c8-31cfc66b8a1f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:07.610026 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1119/2173] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:07 2022] PUT /identity/v3/projects/6940d4d91e2246848bb4a46716aa8e38/users/22766b0421c645cc96078eeff2cda6b9/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:34:07.613893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c6698-8510-4a7d-810e-01758a9c7dc6 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:07.620358 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d5c9c-0eff-4df3-93c8-31cfc66b8a1f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:07.701103 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:07.711338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1d39d-8f9b-41a4-b8cc-689613e6e6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:07.712252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1d39d-8f9b-41a4-b8cc-689613e6e6b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:07.712753 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1d39d-8f9b-41a4-b8cc-689613e6e6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:07.713247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1d39d-8f9b-41a4-b8cc-689613e6e6b6 admin admin] PATH_INFO: `/v3/projects/e60a7a070777440a9427c0b526075b65/users/954189bce80f4974af44adcab2fd8ace/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:07.724390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c6698-8510-4a7d-810e-01758a9c7dc6 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:07.734470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8abdd-a528-4b12-8c63-649470dd4940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:07.735360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8abdd-a528-4b12-8c63-649470dd4940 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:07.735907 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8abdd-a528-4b12-8c63-649470dd4940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:07.736271 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8abdd-a528-4b12-8c63-649470dd4940 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:07.737097 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1d39d-8f9b-41a4-b8cc-689613e6e6b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e60a7a070777440a9427c0b526075b65, user_id=954189bce80f4974af44adcab2fd8ace, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:07.737471 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8abdd-a528-4b12-8c63-649470dd4940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:07.739793 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8abdd-a528-4b12-8c63-649470dd4940 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:07.743281 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1d39d-8f9b-41a4-b8cc-689613e6e6b6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:07.753344 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1120/2174] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:34:07.757412 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8abdd-a528-4b12-8c63-649470dd4940 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:07.761745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1055/2175] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:07 2022] PUT /identity/v3/projects/e60a7a070777440a9427c0b526075b65/users/954189bce80f4974af44adcab2fd8ace/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:07.765791 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1d39d-8f9b-41a4-b8cc-689613e6e6b6 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:07.771838 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8abdd-a528-4b12-8c63-649470dd4940 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:07.855859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1d39d-8f9b-41a4-b8cc-689613e6e6b6 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:07.856975 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77511728-3942-449c-9998-d027226e5cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:07.858272 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77511728-3942-449c-9998-d027226e5cca admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:07.858272 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77511728-3942-449c-9998-d027226e5cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:07.858697 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77511728-3942-449c-9998-d027226e5cca admin admin] PATH_INFO: `/v3/projects/6940d4d91e2246848bb4a46716aa8e38/users/22766b0421c645cc96078eeff2cda6b9/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:07.860948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77511728-3942-449c-9998-d027226e5cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=6940d4d91e2246848bb4a46716aa8e38, user_id=22766b0421c645cc96078eeff2cda6b9, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:07.864033 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77511728-3942-449c-9998-d027226e5cca admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:07.870385 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c184b-28cf-487f-9bbf-a0514bb63050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:07.871058 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c184b-28cf-487f-9bbf-a0514bb63050 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:07.871455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c184b-28cf-487f-9bbf-a0514bb63050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:07.871899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c184b-28cf-487f-9bbf-a0514bb63050 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:07.872707 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c184b-28cf-487f-9bbf-a0514bb63050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:07.874324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c184b-28cf-487f-9bbf-a0514bb63050 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:07.877302 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1121/2176] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:07 2022] PUT /identity/v3/projects/6940d4d91e2246848bb4a46716aa8e38/users/22766b0421c645cc96078eeff2cda6b9/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:34:07.887998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea30c2d-a20f-45cd-9b3c-10404abbd5a5 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:07.888595 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea30c2d-a20f-45cd-9b3c-10404abbd5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:07.888961 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1056/2177] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:34:07.889080 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea30c2d-a20f-45cd-9b3c-10404abbd5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:07.893456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c184b-28cf-487f-9bbf-a0514bb63050 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:07.911963 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c184b-28cf-487f-9bbf-a0514bb63050 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:07.937222 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-6ea30c2d-a20f-45cd-9b3c-10404abbd5a5 None None] MFA Rules not processed for user `22766b0421c645cc96078eeff2cda6b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:07.940257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6ea30c2d-a20f-45cd-9b3c-10404abbd5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:07.982890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:08.008206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0118ee16-36d8-4311-a7f7-99ec63a9eff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:08.009169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118ee16-36d8-4311-a7f7-99ec63a9eff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:08.009430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118ee16-36d8-4311-a7f7-99ec63a9eff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:08.009710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0118ee16-36d8-4311-a7f7-99ec63a9eff4 admin admin] PATH_INFO: `/v3/projects/e60a7a070777440a9427c0b526075b65/users/954189bce80f4974af44adcab2fd8ace/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:08.011871 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0118ee16-36d8-4311-a7f7-99ec63a9eff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e60a7a070777440a9427c0b526075b65, user_id=954189bce80f4974af44adcab2fd8ace, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:08.014623 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0118ee16-36d8-4311-a7f7-99ec63a9eff4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:08.028172 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1057/2178] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:07 2022] PUT /identity/v3/projects/e60a7a070777440a9427c0b526075b65/users/954189bce80f4974af44adcab2fd8ace/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:08.040183 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089833b4-d65a-46bf-92a7-9e8e6626a3c1 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:08.040455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089833b4-d65a-46bf-92a7-9e8e6626a3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:08.040800 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089833b4-d65a-46bf-92a7-9e8e6626a3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:08.054926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1122/2179] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:07 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:08.066611 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea30c2d-a20f-45cd-9b3c-10404abbd5a5 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:08.069312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6ea30c2d-a20f-45cd-9b3c-10404abbd5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:08.077237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-089833b4-d65a-46bf-92a7-9e8e6626a3c1 None None] MFA Rules not processed for user `954189bce80f4974af44adcab2fd8ace`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:08.078315 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-089833b4-d65a-46bf-92a7-9e8e6626a3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:08.141718 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea30c2d-a20f-45cd-9b3c-10404abbd5a5 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:08.162117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:08.191505 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1058/2180] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:34:08.215409 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204ed88-b0d0-4764-9f85-1e80a512dd14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:08.216174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204ed88-b0d0-4764-9f85-1e80a512dd14 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:08.216460 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204ed88-b0d0-4764-9f85-1e80a512dd14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:08.216727 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204ed88-b0d0-4764-9f85-1e80a512dd14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:08.240665 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089833b4-d65a-46bf-92a7-9e8e6626a3c1 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:08.242367 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-089833b4-d65a-46bf-92a7-9e8e6626a3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:08.255967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089833b4-d65a-46bf-92a7-9e8e6626a3c1 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:08.260216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d339f7f-d7bb-45ce-ae49-e1a89d503e10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:08.260835 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d339f7f-d7bb-45ce-ae49-e1a89d503e10 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:08.261112 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d339f7f-d7bb-45ce-ae49-e1a89d503e10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:08.261379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d339f7f-d7bb-45ce-ae49-e1a89d503e10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:08.271579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d339f7f-d7bb-45ce-ae49-e1a89d503e10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:08.273689 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d339f7f-d7bb-45ce-ae49-e1a89d503e10 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:08.293759 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1059/2181] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:08.305274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204ed88-b0d0-4764-9f85-1e80a512dd14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:08.306888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204ed88-b0d0-4764-9f85-1e80a512dd14 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:08.321828 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1123/2182] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:08 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:34:08.358789 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d339f7f-d7bb-45ce-ae49-e1a89d503e10 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:08.376401 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d339f7f-d7bb-45ce-ae49-e1a89d503e10 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:08.385244 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854bb4e-642e-4847-81b8-110103dc8ea1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:08.386119 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854bb4e-642e-4847-81b8-110103dc8ea1 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:08.386517 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854bb4e-642e-4847-81b8-110103dc8ea1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:08.386826 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854bb4e-642e-4847-81b8-110103dc8ea1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:08.422699 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204ed88-b0d0-4764-9f85-1e80a512dd14 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:08.431678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204ed88-b0d0-4764-9f85-1e80a512dd14 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:08.435531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1da0ea-9e64-4d37-8574-92b31c6647d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:08.438227 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1da0ea-9e64-4d37-8574-92b31c6647d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:08.438673 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1da0ea-9e64-4d37-8574-92b31c6647d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:08.439015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1da0ea-9e64-4d37-8574-92b31c6647d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:08.466171 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854bb4e-642e-4847-81b8-110103dc8ea1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:08.467775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854bb4e-642e-4847-81b8-110103dc8ea1 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:08.479391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1060/2183] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:08.516018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1da0ea-9e64-4d37-8574-92b31c6647d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:08.517963 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1da0ea-9e64-4d37-8574-92b31c6647d1 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:08.529906 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1124/2184] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:34:10.831949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854bb4e-642e-4847-81b8-110103dc8ea1 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:10.833399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c854bb4e-642e-4847-81b8-110103dc8ea1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:10.896740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:10.918903 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854bb4e-642e-4847-81b8-110103dc8ea1 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:10.924261 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e8fde-8c66-4c2b-88d6-665db9194263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:10.924907 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e8fde-8c66-4c2b-88d6-665db9194263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:10.925189 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e8fde-8c66-4c2b-88d6-665db9194263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:10.925460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e8fde-8c66-4c2b-88d6-665db9194263 admin admin] PATH_INFO: `/v3/users/45c5cb411b764414a44730891a901bdf` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:10.926739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e8fde-8c66-4c2b-88d6-665db9194263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c5cb411b764414a44730891a901bdf)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:10.929035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e8fde-8c66-4c2b-88d6-665db9194263 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:11.002439 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-758e8fde-8c66-4c2b-88d6-665db9194263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c5cb411b764414a44730891a901bdf'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:11.020576 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-758e8fde-8c66-4c2b-88d6-665db9194263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c5cb411b764414a44730891a901bdf'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:11.033269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-758e8fde-8c66-4c2b-88d6-665db9194263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c5cb411b764414a44730891a901bdf'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:11.034464 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-758e8fde-8c66-4c2b-88d6-665db9194263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c5cb411b764414a44730891a901bdf'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:11.058755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-758e8fde-8c66-4c2b-88d6-665db9194263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c5cb411b764414a44730891a901bdf'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:11.067431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1061/2185] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:10 2022] DELETE /identity/v3/users/45c5cb411b764414a44730891a901bdf => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:11.072510 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9528ec-177f-4464-a910-3226f987c86d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:11.073037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9528ec-177f-4464-a910-3226f987c86d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:11.073137 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9528ec-177f-4464-a910-3226f987c86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:11.100614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ed9528ec-177f-4464-a910-3226f987c86d None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:11.102011 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ed9528ec-177f-4464-a910-3226f987c86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:11.177888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1125/2186] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:11.190998 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e8fde-8c66-4c2b-88d6-665db9194263 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:11.192632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-758e8fde-8c66-4c2b-88d6-665db9194263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:11.338919 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e8fde-8c66-4c2b-88d6-665db9194263 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:11.346726 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7b3f7-d744-4d36-a971-8d0e055b791e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:11.347308 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7b3f7-d744-4d36-a971-8d0e055b791e service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:11.347680 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7b3f7-d744-4d36-a971-8d0e055b791e service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:11.348291 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7b3f7-d744-4d36-a971-8d0e055b791e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:11.373655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7b3f7-d744-4d36-a971-8d0e055b791e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:11.376560 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7b3f7-d744-4d36-a971-8d0e055b791e service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:11.393818 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1062/2187] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:11.900855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9528ec-177f-4464-a910-3226f987c86d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:11.902830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ed9528ec-177f-4464-a910-3226f987c86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:11.932450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9528ec-177f-4464-a910-3226f987c86d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:11.940593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6aeee-f647-4abd-886f-2c4042a5b92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:11.941269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6aeee-f647-4abd-886f-2c4042a5b92d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:11.941561 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6aeee-f647-4abd-886f-2c4042a5b92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:11.942027 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6aeee-f647-4abd-886f-2c4042a5b92d admin admin] PATH_INFO: `/v3/projects/8d1ea3d3f0ac4113bff455db1332a94f` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:11.943834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6aeee-f647-4abd-886f-2c4042a5b92d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d1ea3d3f0ac4113bff455db1332a94f)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:11.948508 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6aeee-f647-4abd-886f-2c4042a5b92d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:11.955510 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:12.024337 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-65a6aeee-f647-4abd-886f-2c4042a5b92d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d1ea3d3f0ac4113bff455db1332a94f', 'request_id': 'req-65a6aeee-f647-4abd-886f-2c4042a5b92d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:12.035512 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-65a6aeee-f647-4abd-886f-2c4042a5b92d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d1ea3d3f0ac4113bff455db1332a94f', 'request_id': 'req-65a6aeee-f647-4abd-886f-2c4042a5b92d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:12.044957 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-65a6aeee-f647-4abd-886f-2c4042a5b92d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d1ea3d3f0ac4113bff455db1332a94f', 'request_id': 'req-65a6aeee-f647-4abd-886f-2c4042a5b92d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:12.059414 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-65a6aeee-f647-4abd-886f-2c4042a5b92d admin admin] The token cache is being invalidate because project 8d1ea3d3f0ac4113bff455db1332a94f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:12.059723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-65a6aeee-f647-4abd-886f-2c4042a5b92d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:12.060430 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-65a6aeee-f647-4abd-886f-2c4042a5b92d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:12.061955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1126/2188] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:11 2022] DELETE /identity/v3/projects/8d1ea3d3f0ac4113bff455db1332a94f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:12.066138 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7b3f7-d744-4d36-a971-8d0e055b791e service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:12.076771 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7b3f7-d744-4d36-a971-8d0e055b791e service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:12.138527 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc3b25-a0d1-41cd-88e3-3868c5311d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:12.139083 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc3b25-a0d1-41cd-88e3-3868c5311d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:12.139365 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc3b25-a0d1-41cd-88e3-3868c5311d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:12.139638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc3b25-a0d1-41cd-88e3-3868c5311d5f admin admin] PATH_INFO: `/v3/users/da281ba9a13b4287a143ed65a642aadb` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:12.140826 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc3b25-a0d1-41cd-88e3-3868c5311d5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=da281ba9a13b4287a143ed65a642aadb)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:12.143139 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc3b25-a0d1-41cd-88e3-3868c5311d5f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:12.153866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:12.200910 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-50fc3b25-a0d1-41cd-88e3-3868c5311d5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da281ba9a13b4287a143ed65a642aadb'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:12.208925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-50fc3b25-a0d1-41cd-88e3-3868c5311d5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da281ba9a13b4287a143ed65a642aadb'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:12.217918 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-50fc3b25-a0d1-41cd-88e3-3868c5311d5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da281ba9a13b4287a143ed65a642aadb'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:12.218813 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-50fc3b25-a0d1-41cd-88e3-3868c5311d5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da281ba9a13b4287a143ed65a642aadb'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:12.237872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-50fc3b25-a0d1-41cd-88e3-3868c5311d5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da281ba9a13b4287a143ed65a642aadb'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:12.245085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1063/2189] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:12 2022] DELETE /identity/v3/users/da281ba9a13b4287a143ed65a642aadb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:12.570501 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6aeee-f647-4abd-886f-2c4042a5b92d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:12.573555 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-65a6aeee-f647-4abd-886f-2c4042a5b92d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:12.590709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6aeee-f647-4abd-886f-2c4042a5b92d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:12.667897 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6576ac7-124d-478a-a032-5c10bf573391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:12.668509 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6576ac7-124d-478a-a032-5c10bf573391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:12.668778 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6576ac7-124d-478a-a032-5c10bf573391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:12.669129 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6576ac7-124d-478a-a032-5c10bf573391 admin admin] PATH_INFO: `/v3/projects/55f63880897a4d2381bc0ee4ae6e2113` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:12.670452 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6576ac7-124d-478a-a032-5c10bf573391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55f63880897a4d2381bc0ee4ae6e2113)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:12.673423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6576ac7-124d-478a-a032-5c10bf573391 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:12.729516 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e6576ac7-124d-478a-a032-5c10bf573391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55f63880897a4d2381bc0ee4ae6e2113', 'request_id': 'req-e6576ac7-124d-478a-a032-5c10bf573391'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:12.737398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e6576ac7-124d-478a-a032-5c10bf573391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55f63880897a4d2381bc0ee4ae6e2113', 'request_id': 'req-e6576ac7-124d-478a-a032-5c10bf573391'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:12.743258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e6576ac7-124d-478a-a032-5c10bf573391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55f63880897a4d2381bc0ee4ae6e2113', 'request_id': 'req-e6576ac7-124d-478a-a032-5c10bf573391'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:12.754560 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e6576ac7-124d-478a-a032-5c10bf573391 admin admin] The token cache is being invalidate because project 55f63880897a4d2381bc0ee4ae6e2113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:12.754849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e6576ac7-124d-478a-a032-5c10bf573391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:12.755512 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e6576ac7-124d-478a-a032-5c10bf573391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:12.757072 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1127/2190] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:12 2022] DELETE /identity/v3/projects/55f63880897a4d2381bc0ee4ae6e2113 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:34:12.762497 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a74c9-6486-466e-af8e-89a098b9bc7d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:12.762777 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a74c9-6486-466e-af8e-89a098b9bc7d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:12.763047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a74c9-6486-466e-af8e-89a098b9bc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:12.783454 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-7a9a74c9-6486-466e-af8e-89a098b9bc7d None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:12.784718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7a9a74c9-6486-466e-af8e-89a098b9bc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:12.843325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1064/2191] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:12.851030 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a40a3-7359-47a2-8051-02582214e348 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:12.851345 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a40a3-7359-47a2-8051-02582214e348 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:12.851649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a40a3-7359-47a2-8051-02582214e348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:12.871769 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-560a40a3-7359-47a2-8051-02582214e348 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:12.873155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-560a40a3-7359-47a2-8051-02582214e348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:12.881474 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1128/2192] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:12.889181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a74c9-6486-466e-af8e-89a098b9bc7d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:12.899605 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a74c9-6486-466e-af8e-89a098b9bc7d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:12.904577 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac017cc1-a0c5-48f0-8346-e962b80621e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:12.905267 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac017cc1-a0c5-48f0-8346-e962b80621e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:12.905446 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac017cc1-a0c5-48f0-8346-e962b80621e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:12.905737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac017cc1-a0c5-48f0-8346-e962b80621e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:12.917840 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac017cc1-a0c5-48f0-8346-e962b80621e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:34:12.918219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac017cc1-a0c5-48f0-8346-e962b80621e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:12.919959 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac017cc1-a0c5-48f0-8346-e962b80621e8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:12.944290 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1065/2193] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:34:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:34:12.949289 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a40a3-7359-47a2-8051-02582214e348 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:12.958579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a40a3-7359-47a2-8051-02582214e348 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:12.963780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c6fbe-8b4c-4f59-a16f-aa117d134b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:12.964424 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c6fbe-8b4c-4f59-a16f-aa117d134b4f admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:12.964702 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c6fbe-8b4c-4f59-a16f-aa117d134b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:12.964973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c6fbe-8b4c-4f59-a16f-aa117d134b4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:12.965598 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c6fbe-8b4c-4f59-a16f-aa117d134b4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:12.967475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c6fbe-8b4c-4f59-a16f-aa117d134b4f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:12.982204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:12.999706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1129/2194] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:12 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:13.000918 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac017cc1-a0c5-48f0-8346-e962b80621e8 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:13.011979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac017cc1-a0c5-48f0-8346-e962b80621e8 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:13.080886 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadb44e-ed64-4321-a25b-af75b13d0465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:13.082327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadb44e-ed64-4321-a25b-af75b13d0465 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:13.082638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadb44e-ed64-4321-a25b-af75b13d0465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:13.082987 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadb44e-ed64-4321-a25b-af75b13d0465 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:13.083625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadb44e-ed64-4321-a25b-af75b13d0465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:13.086226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadb44e-ed64-4321-a25b-af75b13d0465 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:13.108889 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1066/2195] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:12 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:13.113071 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c6fbe-8b4c-4f59-a16f-aa117d134b4f admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:13.123664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c6fbe-8b4c-4f59-a16f-aa117d134b4f admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:13.130186 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418b4b0-2169-4696-aa8d-935988ac38b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:13.131247 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418b4b0-2169-4696-aa8d-935988ac38b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:13.131722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418b4b0-2169-4696-aa8d-935988ac38b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:13.132040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418b4b0-2169-4696-aa8d-935988ac38b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:13.133235 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418b4b0-2169-4696-aa8d-935988ac38b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:13.134857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418b4b0-2169-4696-aa8d-935988ac38b8 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:13.145691 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1130/2196] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:13.150364 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadb44e-ed64-4321-a25b-af75b13d0465 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:13.157264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:13.162101 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadb44e-ed64-4321-a25b-af75b13d0465 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:13.170403 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb331d-e5d5-42e3-bcaa-63cb7636051c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:13.170944 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb331d-e5d5-42e3-bcaa-63cb7636051c admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:13.171221 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb331d-e5d5-42e3-bcaa-63cb7636051c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:13.171490 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb331d-e5d5-42e3-bcaa-63cb7636051c admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:13.172161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb331d-e5d5-42e3-bcaa-63cb7636051c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:13.173750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb331d-e5d5-42e3-bcaa-63cb7636051c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:13.188837 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1067/2197] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:13.190291 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418b4b0-2169-4696-aa8d-935988ac38b8 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:13.203398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418b4b0-2169-4696-aa8d-935988ac38b8 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:13.209155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93800e8f-7759-4382-8bc3-f64ab1bed4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:13.209886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93800e8f-7759-4382-8bc3-f64ab1bed4d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:13.210179 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93800e8f-7759-4382-8bc3-f64ab1bed4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:13.210487 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93800e8f-7759-4382-8bc3-f64ab1bed4d4 admin admin] PATH_INFO: `/v3/projects/6bbbffea79eb40389f9f3b06cf7fb9a3/users/3c6b2b6efe4a4173ab75101785407495/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:13.227754 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93800e8f-7759-4382-8bc3-f64ab1bed4d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bbbffea79eb40389f9f3b06cf7fb9a3, user_id=3c6b2b6efe4a4173ab75101785407495, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:13.230343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93800e8f-7759-4382-8bc3-f64ab1bed4d4 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:13.252242 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1131/2198] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:13 2022] PUT /identity/v3/projects/6bbbffea79eb40389f9f3b06cf7fb9a3/users/3c6b2b6efe4a4173ab75101785407495/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:13.264506 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24656fac-dde0-4f27-9ef3-c9daaab9a331 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:13.264792 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24656fac-dde0-4f27-9ef3-c9daaab9a331 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:13.265068 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24656fac-dde0-4f27-9ef3-c9daaab9a331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:13.294928 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-24656fac-dde0-4f27-9ef3-c9daaab9a331 None None] MFA Rules not processed for user `3c6b2b6efe4a4173ab75101785407495`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:13.296302 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-24656fac-dde0-4f27-9ef3-c9daaab9a331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:13.364448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1068/2199] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:13.370790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93800e8f-7759-4382-8bc3-f64ab1bed4d4 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:13.438162 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93800e8f-7759-4382-8bc3-f64ab1bed4d4 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:13.442887 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37c236-294b-48b9-b6fa-6ee5fe028290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:13.443517 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37c236-294b-48b9-b6fa-6ee5fe028290 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:13.443797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37c236-294b-48b9-b6fa-6ee5fe028290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:13.444111 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37c236-294b-48b9-b6fa-6ee5fe028290 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:13.444755 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37c236-294b-48b9-b6fa-6ee5fe028290 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:13.447310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37c236-294b-48b9-b6fa-6ee5fe028290 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:13.477177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1132/2200] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:13 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:13.481645 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24656fac-dde0-4f27-9ef3-c9daaab9a331 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:13.492981 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24656fac-dde0-4f27-9ef3-c9daaab9a331 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:13.552619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f8f51-34fd-4d69-a08b-265c88a2833f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:13.553626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f8f51-34fd-4d69-a08b-265c88a2833f admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:13.553908 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f8f51-34fd-4d69-a08b-265c88a2833f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:13.554183 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f8f51-34fd-4d69-a08b-265c88a2833f admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:13.555374 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f8f51-34fd-4d69-a08b-265c88a2833f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:13.557321 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f8f51-34fd-4d69-a08b-265c88a2833f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:13.580237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1069/2201] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:13 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:13.584424 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37c236-294b-48b9-b6fa-6ee5fe028290 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:13.603709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37c236-294b-48b9-b6fa-6ee5fe028290 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:13.610226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6b7e8-0f16-469a-a9be-abd8659230e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:13.610854 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6b7e8-0f16-469a-a9be-abd8659230e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:13.611146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6b7e8-0f16-469a-a9be-abd8659230e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:13.611408 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6b7e8-0f16-469a-a9be-abd8659230e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:13.612102 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e6b7e8-0f16-469a-a9be-abd8659230e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:13.613690 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e6b7e8-0f16-469a-a9be-abd8659230e4 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:13.625350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1133/2202] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:13.629740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f8f51-34fd-4d69-a08b-265c88a2833f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:13.640092 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f8f51-34fd-4d69-a08b-265c88a2833f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:13.646510 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfa1ba-73e1-41f3-8ed0-72a72ed47f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:13.647291 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfa1ba-73e1-41f3-8ed0-72a72ed47f5b admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:13.647615 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfa1ba-73e1-41f3-8ed0-72a72ed47f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:13.648029 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfa1ba-73e1-41f3-8ed0-72a72ed47f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:13.648771 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfa1ba-73e1-41f3-8ed0-72a72ed47f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:13.650435 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfa1ba-73e1-41f3-8ed0-72a72ed47f5b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:13.661360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1070/2203] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:13.666090 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6b7e8-0f16-469a-a9be-abd8659230e4 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:13.676797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6b7e8-0f16-469a-a9be-abd8659230e4 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:13.682399 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a1b7e-65a4-4f57-9736-b1fee1559614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:13.683072 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a1b7e-65a4-4f57-9736-b1fee1559614 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:13.683413 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a1b7e-65a4-4f57-9736-b1fee1559614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:13.683693 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a1b7e-65a4-4f57-9736-b1fee1559614 admin admin] PATH_INFO: `/v3/projects/7a720ffcd10b41edb9d30a0bd697653f/users/78442da1a497473d8c3d773c1e624902/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:13.699806 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979a1b7e-65a4-4f57-9736-b1fee1559614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a720ffcd10b41edb9d30a0bd697653f, user_id=78442da1a497473d8c3d773c1e624902, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:13.702428 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979a1b7e-65a4-4f57-9736-b1fee1559614 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:13.715754 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1134/2204] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:13 2022] PUT /identity/v3/projects/7a720ffcd10b41edb9d30a0bd697653f/users/78442da1a497473d8c3d773c1e624902/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:13.726642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc497e8-d324-447e-be1a-ad914a3bc492 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:13.727023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc497e8-d324-447e-be1a-ad914a3bc492 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:13.730673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc497e8-d324-447e-be1a-ad914a3bc492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:13.760673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-edc497e8-d324-447e-be1a-ad914a3bc492 None None] MFA Rules not processed for user `78442da1a497473d8c3d773c1e624902`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:13.761931 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-edc497e8-d324-447e-be1a-ad914a3bc492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:13.813724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:13.834877 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1071/2205] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:13.839443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a1b7e-65a4-4f57-9736-b1fee1559614 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:13.906724 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979a1b7e-65a4-4f57-9736-b1fee1559614 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:13.911500 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db036b-508e-445e-9259-5a91adf05e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:13.912156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db036b-508e-445e-9259-5a91adf05e93 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:13.912442 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db036b-508e-445e-9259-5a91adf05e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:13.912716 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db036b-508e-445e-9259-5a91adf05e93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:13.913354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db036b-508e-445e-9259-5a91adf05e93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:13.916030 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db036b-508e-445e-9259-5a91adf05e93 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:13.934746 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:13.957708 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1135/2206] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:13 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:13.958857 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc497e8-d324-447e-be1a-ad914a3bc492 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:13.977077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc497e8-d324-447e-be1a-ad914a3bc492 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:14.035467 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ddddc-e791-4bce-8917-e5b9989c2fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:14.036260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ddddc-e791-4bce-8917-e5b9989c2fbe admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:14.036540 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ddddc-e791-4bce-8917-e5b9989c2fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:14.036813 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ddddc-e791-4bce-8917-e5b9989c2fbe admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:14.037468 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ddddc-e791-4bce-8917-e5b9989c2fbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:14.040097 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ddddc-e791-4bce-8917-e5b9989c2fbe admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:14.063766 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1072/2207] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:13 2022] POST /identity/v3/users => generated 298 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:14.067974 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db036b-508e-445e-9259-5a91adf05e93 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:14.078922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db036b-508e-445e-9259-5a91adf05e93 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:14.084417 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07776e9-9a3a-4947-857a-65cc3534a3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:14.085026 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07776e9-9a3a-4947-857a-65cc3534a3c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:14.085328 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07776e9-9a3a-4947-857a-65cc3534a3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:14.086092 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07776e9-9a3a-4947-857a-65cc3534a3c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:14.086735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07776e9-9a3a-4947-857a-65cc3534a3c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:14.088405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07776e9-9a3a-4947-857a-65cc3534a3c1 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:14.100893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1136/2208] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:14.105974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ddddc-e791-4bce-8917-e5b9989c2fbe admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:14.116767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ddddc-e791-4bce-8917-e5b9989c2fbe admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:14.121820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222bbe5-8fbd-4293-8771-00afc0c9ddee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:14.122508 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222bbe5-8fbd-4293-8771-00afc0c9ddee admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:14.122753 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222bbe5-8fbd-4293-8771-00afc0c9ddee admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:14.123410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222bbe5-8fbd-4293-8771-00afc0c9ddee admin admin] PATH_INFO: `/v3/projects/4f742cc32a40422ea34d5959731f4444/users/d08a68e5233e4969b591a313763bd225/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:14.139027 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222bbe5-8fbd-4293-8771-00afc0c9ddee admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f742cc32a40422ea34d5959731f4444, user_id=d08a68e5233e4969b591a313763bd225, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:14.141832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222bbe5-8fbd-4293-8771-00afc0c9ddee admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:14.168452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1073/2209] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:14 2022] PUT /identity/v3/projects/4f742cc32a40422ea34d5959731f4444/users/d08a68e5233e4969b591a313763bd225/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:14.175761 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07776e9-9a3a-4947-857a-65cc3534a3c1 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:14.187342 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07776e9-9a3a-4947-857a-65cc3534a3c1 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:14.262557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510945a8-f559-43ad-845f-6d1ec5cc6d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:14.263527 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510945a8-f559-43ad-845f-6d1ec5cc6d46 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:14.263682 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510945a8-f559-43ad-845f-6d1ec5cc6d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:14.264046 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510945a8-f559-43ad-845f-6d1ec5cc6d46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:14.264733 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510945a8-f559-43ad-845f-6d1ec5cc6d46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:14.266400 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510945a8-f559-43ad-845f-6d1ec5cc6d46 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:14.277611 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1137/2210] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:14.281922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222bbe5-8fbd-4293-8771-00afc0c9ddee admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:14.293253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222bbe5-8fbd-4293-8771-00afc0c9ddee admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:14.300397 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813a473-ec71-477c-8654-559c3670a312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:14.301808 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813a473-ec71-477c-8654-559c3670a312 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:14.302448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813a473-ec71-477c-8654-559c3670a312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:14.302993 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813a473-ec71-477c-8654-559c3670a312 admin admin] PATH_INFO: `/v3/projects/4f742cc32a40422ea34d5959731f4444/users/d08a68e5233e4969b591a313763bd225/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:14.305064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813a473-ec71-477c-8654-559c3670a312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f742cc32a40422ea34d5959731f4444, user_id=d08a68e5233e4969b591a313763bd225, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:14.307776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813a473-ec71-477c-8654-559c3670a312 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:14.321736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1074/2211] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:14 2022] PUT /identity/v3/projects/4f742cc32a40422ea34d5959731f4444/users/d08a68e5233e4969b591a313763bd225/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:14.332054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c3af2-4752-4063-a481-085c7680f8f9 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:14.332349 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c3af2-4752-4063-a481-085c7680f8f9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:14.332636 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c3af2-4752-4063-a481-085c7680f8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:14.363911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-063c3af2-4752-4063-a481-085c7680f8f9 None None] MFA Rules not processed for user `d08a68e5233e4969b591a313763bd225`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:14.365153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-063c3af2-4752-4063-a481-085c7680f8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:14.442997 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1138/2212] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:14.455408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813a473-ec71-477c-8654-559c3670a312 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:14.458577 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-3813a473-ec71-477c-8654-559c3670a312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:14.527744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813a473-ec71-477c-8654-559c3670a312 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:14.531545 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f881ea5-f3ad-4122-b5c6-7d345ec3410e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:14.532182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f881ea5-f3ad-4122-b5c6-7d345ec3410e service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:14.532466 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f881ea5-f3ad-4122-b5c6-7d345ec3410e service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:14.532777 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f881ea5-f3ad-4122-b5c6-7d345ec3410e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:14.591942 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f881ea5-f3ad-4122-b5c6-7d345ec3410e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:14.593422 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f881ea5-f3ad-4122-b5c6-7d345ec3410e service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:14.597227 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:14.603731 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1075/2213] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:19.626425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c3af2-4752-4063-a481-085c7680f8f9 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:19.633933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-063c3af2-4752-4063-a481-085c7680f8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:19.737625 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c3af2-4752-4063-a481-085c7680f8f9 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:19.769613 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9dc9c-9e28-40e8-9fc7-97c7c9854a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:19.769613 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9dc9c-9e28-40e8-9fc7-97c7c9854a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:19.770014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9dc9c-9e28-40e8-9fc7-97c7c9854a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:19.770662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9dc9c-9e28-40e8-9fc7-97c7c9854a05 admin admin] PATH_INFO: `/v3/users/076da899d1084121b8d96c8c9da965ce` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:19.772140 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9dc9c-9e28-40e8-9fc7-97c7c9854a05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=076da899d1084121b8d96c8c9da965ce)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:19.774634 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9dc9c-9e28-40e8-9fc7-97c7c9854a05 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:19.780543 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:19.826532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f881ea5-f3ad-4122-b5c6-7d345ec3410e service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:19.827967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0f881ea5-f3ad-4122-b5c6-7d345ec3410e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:19.846355 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f881ea5-f3ad-4122-b5c6-7d345ec3410e service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:19.853844 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7af9dc9c-9e28-40e8-9fc7-97c7c9854a05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '076da899d1084121b8d96c8c9da965ce'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:19.862294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c00440-516b-4b2b-b3ab-e4fd14813c19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:19.863095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c00440-516b-4b2b-b3ab-e4fd14813c19 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:19.863815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7af9dc9c-9e28-40e8-9fc7-97c7c9854a05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '076da899d1084121b8d96c8c9da965ce'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:19.864256 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c00440-516b-4b2b-b3ab-e4fd14813c19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:19.865522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c00440-516b-4b2b-b3ab-e4fd14813c19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:19.889227 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7af9dc9c-9e28-40e8-9fc7-97c7c9854a05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '076da899d1084121b8d96c8c9da965ce'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:19.891960 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7af9dc9c-9e28-40e8-9fc7-97c7c9854a05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '076da899d1084121b8d96c8c9da965ce'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:19.893146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7af9dc9c-9e28-40e8-9fc7-97c7c9854a05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '076da899d1084121b8d96c8c9da965ce'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:19.919751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1139/2214] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:19 2022] DELETE /identity/v3/users/076da899d1084121b8d96c8c9da965ce => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:19.921314 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38513c0-43f9-4d72-bae8-18313deb3ff9 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:19.921816 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38513c0-43f9-4d72-bae8-18313deb3ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:19.922156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38513c0-43f9-4d72-bae8-18313deb3ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:19.958879 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a38513c0-43f9-4d72-bae8-18313deb3ff9 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:19.960214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a38513c0-43f9-4d72-bae8-18313deb3ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:19.976411 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c00440-516b-4b2b-b3ab-e4fd14813c19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:19.978010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c00440-516b-4b2b-b3ab-e4fd14813c19 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:19.988821 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1076/2215] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:34:20.056062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1140/2216] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:34:20.065149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c00440-516b-4b2b-b3ab-e4fd14813c19 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:20.067120 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f8c00440-516b-4b2b-b3ab-e4fd14813c19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:20.183625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c00440-516b-4b2b-b3ab-e4fd14813c19 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:20.264672 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330b5f1-6f39-4ad9-88a2-712dca1e4d94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:20.265396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330b5f1-6f39-4ad9-88a2-712dca1e4d94 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:20.265735 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330b5f1-6f39-4ad9-88a2-712dca1e4d94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:20.266075 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330b5f1-6f39-4ad9-88a2-712dca1e4d94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:20.280885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330b5f1-6f39-4ad9-88a2-712dca1e4d94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:20.283061 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330b5f1-6f39-4ad9-88a2-712dca1e4d94 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:20.297224 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1077/2217] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:20.741392 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38513c0-43f9-4d72-bae8-18313deb3ff9 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:20.742901 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a38513c0-43f9-4d72-bae8-18313deb3ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:20.761048 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38513c0-43f9-4d72-bae8-18313deb3ff9 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:20.767239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803954ac-335b-4316-881e-69272a16ab8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:20.767897 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803954ac-335b-4316-881e-69272a16ab8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:20.768215 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803954ac-335b-4316-881e-69272a16ab8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:20.768489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803954ac-335b-4316-881e-69272a16ab8e admin admin] PATH_INFO: `/v3/projects/93e26c92435045df8b4e76a9f9403cc5` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:20.769606 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803954ac-335b-4316-881e-69272a16ab8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=93e26c92435045df8b4e76a9f9403cc5)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:20.771515 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803954ac-335b-4316-881e-69272a16ab8e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:20.821613 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-803954ac-335b-4316-881e-69272a16ab8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93e26c92435045df8b4e76a9f9403cc5', 'request_id': 'req-803954ac-335b-4316-881e-69272a16ab8e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:20.829381 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-803954ac-335b-4316-881e-69272a16ab8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93e26c92435045df8b4e76a9f9403cc5', 'request_id': 'req-803954ac-335b-4316-881e-69272a16ab8e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:20.835045 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-803954ac-335b-4316-881e-69272a16ab8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93e26c92435045df8b4e76a9f9403cc5', 'request_id': 'req-803954ac-335b-4316-881e-69272a16ab8e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:20.846755 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-803954ac-335b-4316-881e-69272a16ab8e admin admin] The token cache is being invalidate because project 93e26c92435045df8b4e76a9f9403cc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:20.847048 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-803954ac-335b-4316-881e-69272a16ab8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:20.847694 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-803954ac-335b-4316-881e-69272a16ab8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:20.849202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1141/2218] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:20 2022] DELETE /identity/v3/projects/93e26c92435045df8b4e76a9f9403cc5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:20.856196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330b5f1-6f39-4ad9-88a2-712dca1e4d94 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:20.870797 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330b5f1-6f39-4ad9-88a2-712dca1e4d94 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:20.932162 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:20.945896 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1b939-9982-49d9-a794-1f84a6215d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:20.945896 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1b939-9982-49d9-a794-1f84a6215d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:20.945896 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1b939-9982-49d9-a794-1f84a6215d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:20.945896 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1b939-9982-49d9-a794-1f84a6215d5b admin admin] PATH_INFO: `/v3/users/0afc65be55ba48408c556fb2b1e40476` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:20.946499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1b939-9982-49d9-a794-1f84a6215d5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0afc65be55ba48408c556fb2b1e40476)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:20.946781 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1b939-9982-49d9-a794-1f84a6215d5b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:21.003729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4ab1b939-9982-49d9-a794-1f84a6215d5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0afc65be55ba48408c556fb2b1e40476'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:21.016915 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4ab1b939-9982-49d9-a794-1f84a6215d5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0afc65be55ba48408c556fb2b1e40476'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:21.026129 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4ab1b939-9982-49d9-a794-1f84a6215d5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0afc65be55ba48408c556fb2b1e40476'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:21.026734 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4ab1b939-9982-49d9-a794-1f84a6215d5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0afc65be55ba48408c556fb2b1e40476'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:21.043066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4ab1b939-9982-49d9-a794-1f84a6215d5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0afc65be55ba48408c556fb2b1e40476'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:21.052746 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1078/2219] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:20 2022] DELETE /identity/v3/users/0afc65be55ba48408c556fb2b1e40476 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:34:21.389904 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803954ac-335b-4316-881e-69272a16ab8e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:21.391880 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-803954ac-335b-4316-881e-69272a16ab8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:21.405992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803954ac-335b-4316-881e-69272a16ab8e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:21.410012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:21.484186 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3c348-b293-4602-9e91-a4ceba4398b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:21.484833 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3c348-b293-4602-9e91-a4ceba4398b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:21.485111 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3c348-b293-4602-9e91-a4ceba4398b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:21.485382 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3c348-b293-4602-9e91-a4ceba4398b8 admin admin] PATH_INFO: `/v3/projects/b6b2b1737ee24d6f9ef9aa90711ff819` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:21.486537 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3c348-b293-4602-9e91-a4ceba4398b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6b2b1737ee24d6f9ef9aa90711ff819)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:21.488517 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3c348-b293-4602-9e91-a4ceba4398b8 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:21.546409 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-79e3c348-b293-4602-9e91-a4ceba4398b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6b2b1737ee24d6f9ef9aa90711ff819', 'request_id': 'req-79e3c348-b293-4602-9e91-a4ceba4398b8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:21.553355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-79e3c348-b293-4602-9e91-a4ceba4398b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6b2b1737ee24d6f9ef9aa90711ff819', 'request_id': 'req-79e3c348-b293-4602-9e91-a4ceba4398b8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:21.559866 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-79e3c348-b293-4602-9e91-a4ceba4398b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6b2b1737ee24d6f9ef9aa90711ff819', 'request_id': 'req-79e3c348-b293-4602-9e91-a4ceba4398b8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:21.572076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-79e3c348-b293-4602-9e91-a4ceba4398b8 admin admin] The token cache is being invalidate because project b6b2b1737ee24d6f9ef9aa90711ff819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:21.572370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-79e3c348-b293-4602-9e91-a4ceba4398b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:21.573048 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-79e3c348-b293-4602-9e91-a4ceba4398b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:21.574572 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1142/2220] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:21 2022] DELETE /identity/v3/projects/b6b2b1737ee24d6f9ef9aa90711ff819 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:21.578746 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1b939-9982-49d9-a794-1f84a6215d5b admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:21.580229 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4ab1b939-9982-49d9-a794-1f84a6215d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:21.593333 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1b939-9982-49d9-a794-1f84a6215d5b admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:21.652301 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9fe3a3-e052-46a0-951d-238b788a9e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:21.652941 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fe3a3-e052-46a0-951d-238b788a9e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:21.653224 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fe3a3-e052-46a0-951d-238b788a9e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:21.653502 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fe3a3-e052-46a0-951d-238b788a9e26 admin admin] PATH_INFO: `/v3/users/954189bce80f4974af44adcab2fd8ace` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:21.654659 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9fe3a3-e052-46a0-951d-238b788a9e26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=954189bce80f4974af44adcab2fd8ace)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:21.656828 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9fe3a3-e052-46a0-951d-238b788a9e26 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:21.733574 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ac9fe3a3-e052-46a0-951d-238b788a9e26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '954189bce80f4974af44adcab2fd8ace'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:21.748839 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ac9fe3a3-e052-46a0-951d-238b788a9e26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '954189bce80f4974af44adcab2fd8ace'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:21.759003 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ac9fe3a3-e052-46a0-951d-238b788a9e26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '954189bce80f4974af44adcab2fd8ace'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:21.759914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ac9fe3a3-e052-46a0-951d-238b788a9e26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '954189bce80f4974af44adcab2fd8ace'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:21.790178 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:21.795264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ac9fe3a3-e052-46a0-951d-238b788a9e26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '954189bce80f4974af44adcab2fd8ace'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:21.812739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1079/2221] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:21 2022] DELETE /identity/v3/users/954189bce80f4974af44adcab2fd8ace => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:22.160450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3c348-b293-4602-9e91-a4ceba4398b8 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:22.163102 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-79e3c348-b293-4602-9e91-a4ceba4398b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:22.194453 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3c348-b293-4602-9e91-a4ceba4398b8 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:22.298846 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e0daf-0dd9-4ef3-922f-918e7b7fa712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:22.299513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e0daf-0dd9-4ef3-922f-918e7b7fa712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:22.299795 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e0daf-0dd9-4ef3-922f-918e7b7fa712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:22.300107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e0daf-0dd9-4ef3-922f-918e7b7fa712 admin admin] PATH_INFO: `/v3/projects/e60a7a070777440a9427c0b526075b65` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:22.301270 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e0daf-0dd9-4ef3-922f-918e7b7fa712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e60a7a070777440a9427c0b526075b65)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:22.303188 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e0daf-0dd9-4ef3-922f-918e7b7fa712 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:22.355064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:22.367053 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-2c2e0daf-0dd9-4ef3-922f-918e7b7fa712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e60a7a070777440a9427c0b526075b65', 'request_id': 'req-2c2e0daf-0dd9-4ef3-922f-918e7b7fa712'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:22.380463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-2c2e0daf-0dd9-4ef3-922f-918e7b7fa712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e60a7a070777440a9427c0b526075b65', 'request_id': 'req-2c2e0daf-0dd9-4ef3-922f-918e7b7fa712'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:22.387827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-2c2e0daf-0dd9-4ef3-922f-918e7b7fa712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e60a7a070777440a9427c0b526075b65', 'request_id': 'req-2c2e0daf-0dd9-4ef3-922f-918e7b7fa712'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:22.405218 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-2c2e0daf-0dd9-4ef3-922f-918e7b7fa712 admin admin] The token cache is being invalidate because project e60a7a070777440a9427c0b526075b65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:22.405689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-2c2e0daf-0dd9-4ef3-922f-918e7b7fa712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:22.406971 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-2c2e0daf-0dd9-4ef3-922f-918e7b7fa712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:22.416070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1143/2222] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:22 2022] DELETE /identity/v3/projects/e60a7a070777440a9427c0b526075b65 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:34:22.418785 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b702a19-2b75-4d42-a5b5-173f2ed8ca93 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:22.419499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b702a19-2b75-4d42-a5b5-173f2ed8ca93 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:22.419638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b702a19-2b75-4d42-a5b5-173f2ed8ca93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:22.439497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e0daf-0dd9-4ef3-922f-918e7b7fa712 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:22.443537 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2c2e0daf-0dd9-4ef3-922f-918e7b7fa712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:22.445056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6b702a19-2b75-4d42-a5b5-173f2ed8ca93 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:22.446848 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6b702a19-2b75-4d42-a5b5-173f2ed8ca93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:22.523067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e0daf-0dd9-4ef3-922f-918e7b7fa712 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:22.528234 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e5666-73da-42e7-9948-ba40001d7317 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:22.528860 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e5666-73da-42e7-9948-ba40001d7317 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:22.529157 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e5666-73da-42e7-9948-ba40001d7317 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:22.529494 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e5666-73da-42e7-9948-ba40001d7317 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:22.531705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-031e5666-73da-42e7-9948-ba40001d7317 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:22.554386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1080/2223] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:34:22.564337 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d580e-44e5-451f-8c25-668aa178d183 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:22.564670 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d580e-44e5-451f-8c25-668aa178d183 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:22.564948 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d580e-44e5-451f-8c25-668aa178d183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:22.589968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-2c0d580e-44e5-451f-8c25-668aa178d183 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:22.591228 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2c0d580e-44e5-451f-8c25-668aa178d183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:22.599694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1081/2224] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:34:22.605585 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d580e-44e5-451f-8c25-668aa178d183 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:22.615283 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d580e-44e5-451f-8c25-668aa178d183 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:22.619269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e5666-73da-42e7-9948-ba40001d7317 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:22.620865 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e5666-73da-42e7-9948-ba40001d7317 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:22.624827 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb7f77-df48-471e-8b3c-a112e491b7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:22.626019 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb7f77-df48-471e-8b3c-a112e491b7c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:22.626384 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb7f77-df48-471e-8b3c-a112e491b7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:22.626823 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb7f77-df48-471e-8b3c-a112e491b7c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:22.627529 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb7f77-df48-471e-8b3c-a112e491b7c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:34:22.628013 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb7f77-df48-471e-8b3c-a112e491b7c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:22.630166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb7f77-df48-471e-8b3c-a112e491b7c7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:22.636159 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1144/2225] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:22 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:22.643982 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1082/2226] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:34:22.648398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e5666-73da-42e7-9948-ba40001d7317 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:22.661091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e5666-73da-42e7-9948-ba40001d7317 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:22.666770 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62688f04-d334-4018-92b7-42c969318019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:22.667111 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62688f04-d334-4018-92b7-42c969318019 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:22.667457 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62688f04-d334-4018-92b7-42c969318019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:22.667778 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62688f04-d334-4018-92b7-42c969318019 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:22.668528 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62688f04-d334-4018-92b7-42c969318019 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:22.670589 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62688f04-d334-4018-92b7-42c969318019 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:22.707222 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1145/2227] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:22 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:22.712657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb7f77-df48-471e-8b3c-a112e491b7c7 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:22.723238 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb7f77-df48-471e-8b3c-a112e491b7c7 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:22.809368 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b87ca-611b-4fd4-997c-62c7fb05dcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:22.810686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b87ca-611b-4fd4-997c-62c7fb05dcf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:22.811135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b87ca-611b-4fd4-997c-62c7fb05dcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:22.811595 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b87ca-611b-4fd4-997c-62c7fb05dcf1 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:22.812677 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b87ca-611b-4fd4-997c-62c7fb05dcf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:22.816039 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b87ca-611b-4fd4-997c-62c7fb05dcf1 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:22.855988 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1083/2228] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:22 2022] POST /identity/v3/users => generated 290 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:34:22.857617 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62688f04-d334-4018-92b7-42c969318019 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:22.899562 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62688f04-d334-4018-92b7-42c969318019 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:22.908409 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfe0b3-9dc8-46d4-8968-b146125f7508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:22.909167 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfe0b3-9dc8-46d4-8968-b146125f7508 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:22.909559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfe0b3-9dc8-46d4-8968-b146125f7508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:22.909743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfe0b3-9dc8-46d4-8968-b146125f7508 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:22.910436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bfe0b3-9dc8-46d4-8968-b146125f7508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:22.912180 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bfe0b3-9dc8-46d4-8968-b146125f7508 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:22.935534 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1146/2229] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:22.937906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b87ca-611b-4fd4-997c-62c7fb05dcf1 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:22.958831 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b87ca-611b-4fd4-997c-62c7fb05dcf1 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:22.968167 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce8845-2e5a-4561-862d-399775439cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:22.968802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce8845-2e5a-4561-862d-399775439cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:22.969078 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce8845-2e5a-4561-862d-399775439cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:22.969387 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce8845-2e5a-4561-862d-399775439cd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:22.970229 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce8845-2e5a-4561-862d-399775439cd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:22.972414 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce8845-2e5a-4561-862d-399775439cd1 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:22.989081 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1084/2230] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:22.993940 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfe0b3-9dc8-46d4-8968-b146125f7508 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:23.009990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfe0b3-9dc8-46d4-8968-b146125f7508 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:23.015382 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0a59f-c4de-466e-8973-23c374c61860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:23.017223 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0a59f-c4de-466e-8973-23c374c61860 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:23.017795 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0a59f-c4de-466e-8973-23c374c61860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:23.018300 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf0a59f-c4de-466e-8973-23c374c61860 admin admin] PATH_INFO: `/v3/projects/e40f865449894de498fc099421ee5bea/users/8389ce6324874441a1ee48feeee611b4/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:23.048630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0a59f-c4de-466e-8973-23c374c61860 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e40f865449894de498fc099421ee5bea, user_id=8389ce6324874441a1ee48feeee611b4, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:23.048630 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf0a59f-c4de-466e-8973-23c374c61860 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:23.071624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1147/2231] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:22 2022] PUT /identity/v3/projects/e40f865449894de498fc099421ee5bea/users/8389ce6324874441a1ee48feeee611b4/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:23.083403 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78412a06-6fda-4b41-bf25-04c146718384 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:23.083758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78412a06-6fda-4b41-bf25-04c146718384 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:23.084066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78412a06-6fda-4b41-bf25-04c146718384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:23.158069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-78412a06-6fda-4b41-bf25-04c146718384 None None] MFA Rules not processed for user `8389ce6324874441a1ee48feeee611b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:23.160614 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-78412a06-6fda-4b41-bf25-04c146718384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:23.229828 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:23.262377 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1085/2232] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:23.269651 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0a59f-c4de-466e-8973-23c374c61860 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:23.296563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:23.346585 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf0a59f-c4de-466e-8973-23c374c61860 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:23.352994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d6dcc-c9c7-4c6b-8135-02defb0a0304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:23.353612 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d6dcc-c9c7-4c6b-8135-02defb0a0304 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:23.353897 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d6dcc-c9c7-4c6b-8135-02defb0a0304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:23.354175 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d6dcc-c9c7-4c6b-8135-02defb0a0304 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:23.354827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d6dcc-c9c7-4c6b-8135-02defb0a0304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:23.356767 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d6dcc-c9c7-4c6b-8135-02defb0a0304 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:23.392068 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1148/2233] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:23 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:23.398652 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78412a06-6fda-4b41-bf25-04c146718384 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:23.413740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78412a06-6fda-4b41-bf25-04c146718384 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:23.487302 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81edf21-f154-41c4-b39d-23d6ba66d7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:23.488204 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81edf21-f154-41c4-b39d-23d6ba66d7f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:23.488204 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81edf21-f154-41c4-b39d-23d6ba66d7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:23.488481 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81edf21-f154-41c4-b39d-23d6ba66d7f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:23.489110 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81edf21-f154-41c4-b39d-23d6ba66d7f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:23.491025 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81edf21-f154-41c4-b39d-23d6ba66d7f1 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:23.515395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1086/2234] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:23 2022] POST /identity/v3/users => generated 297 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:23.519551 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d6dcc-c9c7-4c6b-8135-02defb0a0304 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:23.535572 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d6dcc-c9c7-4c6b-8135-02defb0a0304 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:23.541175 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13045d0f-460d-4755-ae30-3dda53ae54c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:23.541829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13045d0f-460d-4755-ae30-3dda53ae54c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:23.542113 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13045d0f-460d-4755-ae30-3dda53ae54c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:23.542391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13045d0f-460d-4755-ae30-3dda53ae54c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:23.543043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13045d0f-460d-4755-ae30-3dda53ae54c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:23.544680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13045d0f-460d-4755-ae30-3dda53ae54c0 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:23.557154 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1149/2235] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:23.561335 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81edf21-f154-41c4-b39d-23d6ba66d7f1 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:23.576599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81edf21-f154-41c4-b39d-23d6ba66d7f1 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:23.583182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe678ff-ac18-4436-a46b-545e682820f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:23.583877 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe678ff-ac18-4436-a46b-545e682820f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:23.584187 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe678ff-ac18-4436-a46b-545e682820f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:23.584469 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe678ff-ac18-4436-a46b-545e682820f0 admin admin] PATH_INFO: `/v3/projects/ff22a9f078014e7dac64d3356b03ac31/users/a59a7338af9444f7a6196f3fd74b08c3/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:23.612556 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe678ff-ac18-4436-a46b-545e682820f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff22a9f078014e7dac64d3356b03ac31, user_id=a59a7338af9444f7a6196f3fd74b08c3, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:23.615244 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe678ff-ac18-4436-a46b-545e682820f0 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:23.630988 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1087/2236] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:23 2022] PUT /identity/v3/projects/ff22a9f078014e7dac64d3356b03ac31/users/a59a7338af9444f7a6196f3fd74b08c3/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:23.635550 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13045d0f-460d-4755-ae30-3dda53ae54c0 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:23.646913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13045d0f-460d-4755-ae30-3dda53ae54c0 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:23.719032 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e3338-6020-4832-8232-959096052756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:23.720002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e3338-6020-4832-8232-959096052756 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:23.720002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e3338-6020-4832-8232-959096052756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:23.720229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e3338-6020-4832-8232-959096052756 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:23.720940 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e3338-6020-4832-8232-959096052756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:23.722534 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e3338-6020-4832-8232-959096052756 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:23.734439 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1150/2237] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:23.740560 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe678ff-ac18-4436-a46b-545e682820f0 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:23.755310 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe678ff-ac18-4436-a46b-545e682820f0 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:23.761199 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd21872-b36d-469d-93bd-49a3d21d665f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:23.761870 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd21872-b36d-469d-93bd-49a3d21d665f admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:23.762148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd21872-b36d-469d-93bd-49a3d21d665f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:23.762422 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd21872-b36d-469d-93bd-49a3d21d665f admin admin] PATH_INFO: `/v3/projects/ff22a9f078014e7dac64d3356b03ac31/users/a59a7338af9444f7a6196f3fd74b08c3/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:23.764379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd21872-b36d-469d-93bd-49a3d21d665f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff22a9f078014e7dac64d3356b03ac31, user_id=a59a7338af9444f7a6196f3fd74b08c3, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:23.767026 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd21872-b36d-469d-93bd-49a3d21d665f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:23.778577 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1088/2238] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:23 2022] PUT /identity/v3/projects/ff22a9f078014e7dac64d3356b03ac31/users/a59a7338af9444f7a6196f3fd74b08c3/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:23.791651 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b347693-5b91-4d4a-8951-4b5bc5ea462a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:23.791651 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b347693-5b91-4d4a-8951-4b5bc5ea462a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:23.791651 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b347693-5b91-4d4a-8951-4b5bc5ea462a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:23.826739 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-9b347693-5b91-4d4a-8951-4b5bc5ea462a None None] MFA Rules not processed for user `a59a7338af9444f7a6196f3fd74b08c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:23.828024 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9b347693-5b91-4d4a-8951-4b5bc5ea462a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:23.904754 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1151/2239] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:34:23.916156 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd21872-b36d-469d-93bd-49a3d21d665f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:23.989622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd21872-b36d-469d-93bd-49a3d21d665f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:23.996259 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8685da-05a7-46b7-bda0-88f8159ee9b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:23.997442 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8685da-05a7-46b7-bda0-88f8159ee9b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:23.997769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8685da-05a7-46b7-bda0-88f8159ee9b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:23.998604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8685da-05a7-46b7-bda0-88f8159ee9b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:24.012000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8685da-05a7-46b7-bda0-88f8159ee9b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:24.013454 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8685da-05a7-46b7-bda0-88f8159ee9b8 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:24.024897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1089/2240] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:24.594549 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b347693-5b91-4d4a-8951-4b5bc5ea462a None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:24.621231 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:24.689975 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b347693-5b91-4d4a-8951-4b5bc5ea462a None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:24.696888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9a951-bec8-4ecf-b201-a3199c230c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:24.697782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9a951-bec8-4ecf-b201-a3199c230c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:24.698062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9a951-bec8-4ecf-b201-a3199c230c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:24.698343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9a951-bec8-4ecf-b201-a3199c230c8c admin admin] PATH_INFO: `/v3/users/8389ce6324874441a1ee48feeee611b4` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:24.699595 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9a951-bec8-4ecf-b201-a3199c230c8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8389ce6324874441a1ee48feeee611b4)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:24.701603 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9a951-bec8-4ecf-b201-a3199c230c8c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:24.776428 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-04f9a951-bec8-4ecf-b201-a3199c230c8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8389ce6324874441a1ee48feeee611b4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:24.784912 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-04f9a951-bec8-4ecf-b201-a3199c230c8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8389ce6324874441a1ee48feeee611b4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:24.795221 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-04f9a951-bec8-4ecf-b201-a3199c230c8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8389ce6324874441a1ee48feeee611b4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:24.796022 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-04f9a951-bec8-4ecf-b201-a3199c230c8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8389ce6324874441a1ee48feeee611b4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:24.796734 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-04f9a951-bec8-4ecf-b201-a3199c230c8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8389ce6324874441a1ee48feeee611b4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:24.825198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1152/2241] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:24 2022] DELETE /identity/v3/users/8389ce6324874441a1ee48feeee611b4 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:24.829731 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d83fd-cd3d-4216-8478-aace726d492b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:24.830195 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d83fd-cd3d-4216-8478-aace726d492b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:24.830195 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d83fd-cd3d-4216-8478-aace726d492b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:24.866482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-193d83fd-cd3d-4216-8478-aace726d492b None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:24.868484 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-193d83fd-cd3d-4216-8478-aace726d492b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:24.907390 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:24.948078 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1090/2242] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:24.961749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9a951-bec8-4ecf-b201-a3199c230c8c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:24.963655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-04f9a951-bec8-4ecf-b201-a3199c230c8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:25.112272 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9a951-bec8-4ecf-b201-a3199c230c8c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:25.122763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a54fe4-d2e5-407d-9140-aa15dbdb41ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:25.124385 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a54fe4-d2e5-407d-9140-aa15dbdb41ec service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:25.124981 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a54fe4-d2e5-407d-9140-aa15dbdb41ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:25.125743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a54fe4-d2e5-407d-9140-aa15dbdb41ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:25.147861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a54fe4-d2e5-407d-9140-aa15dbdb41ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:25.152889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a54fe4-d2e5-407d-9140-aa15dbdb41ec service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:25.176558 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1153/2243] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:25.275390 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d83fd-cd3d-4216-8478-aace726d492b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:25.279982 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-193d83fd-cd3d-4216-8478-aace726d492b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:25.322122 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d83fd-cd3d-4216-8478-aace726d492b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:25.335600 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561011e5-46bc-4f09-ae13-8b9c7c243bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:25.336987 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561011e5-46bc-4f09-ae13-8b9c7c243bd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:25.337410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561011e5-46bc-4f09-ae13-8b9c7c243bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:25.338404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561011e5-46bc-4f09-ae13-8b9c7c243bd9 admin admin] PATH_INFO: `/v3/users/19c00cca09944f70914d73eb51ee879a` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:25.341578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561011e5-46bc-4f09-ae13-8b9c7c243bd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19c00cca09944f70914d73eb51ee879a)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:25.344520 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561011e5-46bc-4f09-ae13-8b9c7c243bd9 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:25.471985 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-561011e5-46bc-4f09-ae13-8b9c7c243bd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19c00cca09944f70914d73eb51ee879a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:25.488298 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-561011e5-46bc-4f09-ae13-8b9c7c243bd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19c00cca09944f70914d73eb51ee879a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:25.502172 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-561011e5-46bc-4f09-ae13-8b9c7c243bd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19c00cca09944f70914d73eb51ee879a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:25.503323 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-561011e5-46bc-4f09-ae13-8b9c7c243bd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19c00cca09944f70914d73eb51ee879a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:25.528131 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-561011e5-46bc-4f09-ae13-8b9c7c243bd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19c00cca09944f70914d73eb51ee879a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:25.550826 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1091/2244] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:25 2022] DELETE /identity/v3/users/19c00cca09944f70914d73eb51ee879a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:25.557069 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c75cf-b81c-4fd6-9656-79e82758709c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:25.559680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c75cf-b81c-4fd6-9656-79e82758709c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:25.560066 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c75cf-b81c-4fd6-9656-79e82758709c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:25.590611 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b21c75cf-b81c-4fd6-9656-79e82758709c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:25.592234 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b21c75cf-b81c-4fd6-9656-79e82758709c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:25.652056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:25.714444 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1154/2245] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:25.735668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561011e5-46bc-4f09-ae13-8b9c7c243bd9 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:25.744001 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-561011e5-46bc-4f09-ae13-8b9c7c243bd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:25.763822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c75cf-b81c-4fd6-9656-79e82758709c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:25.765453 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b21c75cf-b81c-4fd6-9656-79e82758709c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:25.801865 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c75cf-b81c-4fd6-9656-79e82758709c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:25.821232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee7284-5a10-4c98-bfab-1859ae23bef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:25.821906 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee7284-5a10-4c98-bfab-1859ae23bef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:25.822193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee7284-5a10-4c98-bfab-1859ae23bef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:25.823038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee7284-5a10-4c98-bfab-1859ae23bef1 admin admin] PATH_INFO: `/v3/projects/e40f865449894de498fc099421ee5bea` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:25.823746 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee7284-5a10-4c98-bfab-1859ae23bef1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e40f865449894de498fc099421ee5bea)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:25.825758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee7284-5a10-4c98-bfab-1859ae23bef1 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:25.863834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561011e5-46bc-4f09-ae13-8b9c7c243bd9 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:25.870096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c7efd-8827-4a1d-a6bf-71f8443e4ae3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:25.870991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c7efd-8827-4a1d-a6bf-71f8443e4ae3 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:25.871684 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c7efd-8827-4a1d-a6bf-71f8443e4ae3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:25.872144 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c7efd-8827-4a1d-a6bf-71f8443e4ae3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:25.885271 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c7efd-8827-4a1d-a6bf-71f8443e4ae3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:25.887287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c7efd-8827-4a1d-a6bf-71f8443e4ae3 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:25.901865 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1092/2246] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:25.903286 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-75ee7284-5a10-4c98-bfab-1859ae23bef1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e40f865449894de498fc099421ee5bea', 'request_id': 'req-75ee7284-5a10-4c98-bfab-1859ae23bef1'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:25.912533 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-75ee7284-5a10-4c98-bfab-1859ae23bef1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e40f865449894de498fc099421ee5bea', 'request_id': 'req-75ee7284-5a10-4c98-bfab-1859ae23bef1'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:25.920229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-75ee7284-5a10-4c98-bfab-1859ae23bef1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e40f865449894de498fc099421ee5bea', 'request_id': 'req-75ee7284-5a10-4c98-bfab-1859ae23bef1'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:25.939148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-75ee7284-5a10-4c98-bfab-1859ae23bef1 admin admin] The token cache is being invalidate because project e40f865449894de498fc099421ee5bea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:25.939473 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-75ee7284-5a10-4c98-bfab-1859ae23bef1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:25.940312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-75ee7284-5a10-4c98-bfab-1859ae23bef1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:25.942166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1155/2247] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:25 2022] DELETE /identity/v3/projects/e40f865449894de498fc099421ee5bea => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:25.946190 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c7efd-8827-4a1d-a6bf-71f8443e4ae3 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:25.958032 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c7efd-8827-4a1d-a6bf-71f8443e4ae3 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:26.034313 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:26.042922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6707018-9d7b-4cf5-950f-a0b0a9e0b0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:26.043786 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6707018-9d7b-4cf5-950f-a0b0a9e0b0a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:26.044005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6707018-9d7b-4cf5-950f-a0b0a9e0b0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:26.044325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6707018-9d7b-4cf5-950f-a0b0a9e0b0a8 admin admin] PATH_INFO: `/v3/users/a59a7338af9444f7a6196f3fd74b08c3` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:26.045981 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6707018-9d7b-4cf5-950f-a0b0a9e0b0a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a59a7338af9444f7a6196f3fd74b08c3)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:26.049007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6707018-9d7b-4cf5-950f-a0b0a9e0b0a8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:26.148371 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f6707018-9d7b-4cf5-950f-a0b0a9e0b0a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a59a7338af9444f7a6196f3fd74b08c3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:26.158046 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f6707018-9d7b-4cf5-950f-a0b0a9e0b0a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a59a7338af9444f7a6196f3fd74b08c3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:26.173633 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f6707018-9d7b-4cf5-950f-a0b0a9e0b0a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a59a7338af9444f7a6196f3fd74b08c3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:26.174904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f6707018-9d7b-4cf5-950f-a0b0a9e0b0a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a59a7338af9444f7a6196f3fd74b08c3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:26.190216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f6707018-9d7b-4cf5-950f-a0b0a9e0b0a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a59a7338af9444f7a6196f3fd74b08c3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:26.196611 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1093/2248] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:25 2022] DELETE /identity/v3/users/a59a7338af9444f7a6196f3fd74b08c3 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:26.556090 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee7284-5a10-4c98-bfab-1859ae23bef1 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:26.557559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-75ee7284-5a10-4c98-bfab-1859ae23bef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:26.572388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee7284-5a10-4c98-bfab-1859ae23bef1 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:26.656279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b05f1b-cdb2-4d79-a1f8-b291e3666e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:26.657650 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b05f1b-cdb2-4d79-a1f8-b291e3666e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:26.658088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b05f1b-cdb2-4d79-a1f8-b291e3666e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:26.658514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b05f1b-cdb2-4d79-a1f8-b291e3666e28 admin admin] PATH_INFO: `/v3/projects/7c69548e9cee495382b398a072732fd9` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:26.659907 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b05f1b-cdb2-4d79-a1f8-b291e3666e28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c69548e9cee495382b398a072732fd9)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:26.663841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b05f1b-cdb2-4d79-a1f8-b291e3666e28 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:26.693816 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6707018-9d7b-4cf5-950f-a0b0a9e0b0a8 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:26.695426 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f6707018-9d7b-4cf5-950f-a0b0a9e0b0a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:26.735232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6707018-9d7b-4cf5-950f-a0b0a9e0b0a8 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:26.750621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b8b05f1b-cdb2-4d79-a1f8-b291e3666e28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c69548e9cee495382b398a072732fd9', 'request_id': 'req-b8b05f1b-cdb2-4d79-a1f8-b291e3666e28'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:26.765433 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b8b05f1b-cdb2-4d79-a1f8-b291e3666e28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c69548e9cee495382b398a072732fd9', 'request_id': 'req-b8b05f1b-cdb2-4d79-a1f8-b291e3666e28'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:26.770882 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:26.772558 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b8b05f1b-cdb2-4d79-a1f8-b291e3666e28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c69548e9cee495382b398a072732fd9', 'request_id': 'req-b8b05f1b-cdb2-4d79-a1f8-b291e3666e28'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:26.788363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b8b05f1b-cdb2-4d79-a1f8-b291e3666e28 admin admin] The token cache is being invalidate because project 7c69548e9cee495382b398a072732fd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:26.788895 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b8b05f1b-cdb2-4d79-a1f8-b291e3666e28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:26.790249 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b8b05f1b-cdb2-4d79-a1f8-b291e3666e28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:26.792594 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1156/2249] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:26 2022] DELETE /identity/v3/projects/7c69548e9cee495382b398a072732fd9 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:34:26.796232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b05f1b-cdb2-4d79-a1f8-b291e3666e28 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:26.797893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b8b05f1b-cdb2-4d79-a1f8-b291e3666e28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:26.833552 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb49a0-9985-49f8-9d5f-73d9d07c0553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:26.834209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb49a0-9985-49f8-9d5f-73d9d07c0553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:26.834489 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb49a0-9985-49f8-9d5f-73d9d07c0553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:26.834768 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb49a0-9985-49f8-9d5f-73d9d07c0553 admin admin] PATH_INFO: `/v3/projects/ff22a9f078014e7dac64d3356b03ac31` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:26.836035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb49a0-9985-49f8-9d5f-73d9d07c0553 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff22a9f078014e7dac64d3356b03ac31)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:26.838081 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdb49a0-9985-49f8-9d5f-73d9d07c0553 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:26.879123 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b05f1b-cdb2-4d79-a1f8-b291e3666e28 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:26.891250 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-dfdb49a0-9985-49f8-9d5f-73d9d07c0553 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff22a9f078014e7dac64d3356b03ac31', 'request_id': 'req-dfdb49a0-9985-49f8-9d5f-73d9d07c0553'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:26.897868 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-dfdb49a0-9985-49f8-9d5f-73d9d07c0553 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff22a9f078014e7dac64d3356b03ac31', 'request_id': 'req-dfdb49a0-9985-49f8-9d5f-73d9d07c0553'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:26.908474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-dfdb49a0-9985-49f8-9d5f-73d9d07c0553 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff22a9f078014e7dac64d3356b03ac31', 'request_id': 'req-dfdb49a0-9985-49f8-9d5f-73d9d07c0553'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:26.923899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-dfdb49a0-9985-49f8-9d5f-73d9d07c0553 admin admin] The token cache is being invalidate because project ff22a9f078014e7dac64d3356b03ac31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:26.924311 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-dfdb49a0-9985-49f8-9d5f-73d9d07c0553 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:26.925748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-dfdb49a0-9985-49f8-9d5f-73d9d07c0553 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:26.928063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1094/2250] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:26 2022] DELETE /identity/v3/projects/ff22a9f078014e7dac64d3356b03ac31 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:34:26.935846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc65f4-c1f3-49e4-b0d5-7726dfcc75e3 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:26.936296 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc65f4-c1f3-49e4-b0d5-7726dfcc75e3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:26.936876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc65f4-c1f3-49e4-b0d5-7726dfcc75e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:26.947312 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:26.974700 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-badc65f4-c1f3-49e4-b0d5-7726dfcc75e3 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:26.975868 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-badc65f4-c1f3-49e4-b0d5-7726dfcc75e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:26.977647 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c7ba5-cc2e-4bf5-b5a6-a499ec619aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:26.978587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c7ba5-cc2e-4bf5-b5a6-a499ec619aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:26.979078 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c7ba5-cc2e-4bf5-b5a6-a499ec619aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:26.979475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c7ba5-cc2e-4bf5-b5a6-a499ec619aa6 admin admin] PATH_INFO: `/v3/users/22766b0421c645cc96078eeff2cda6b9` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:26.981257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c7ba5-cc2e-4bf5-b5a6-a499ec619aa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22766b0421c645cc96078eeff2cda6b9)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:26.983571 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c7ba5-cc2e-4bf5-b5a6-a499ec619aa6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:26.990950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1095/2251] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:27.005684 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94113833-b73b-4c99-9823-4874fd52895f None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:27.005982 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94113833-b73b-4c99-9823-4874fd52895f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:27.006247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94113833-b73b-4c99-9823-4874fd52895f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:27.047609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-94113833-b73b-4c99-9823-4874fd52895f None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:27.048886 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-94113833-b73b-4c99-9823-4874fd52895f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:27.058027 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1096/2252] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:27.062514 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94113833-b73b-4c99-9823-4874fd52895f None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:27.078734 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94113833-b73b-4c99-9823-4874fd52895f None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:27.085003 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ecc5e-b525-4294-89aa-a9e59390d245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:27.085638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ecc5e-b525-4294-89aa-a9e59390d245 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:27.085917 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ecc5e-b525-4294-89aa-a9e59390d245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:27.086191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ecc5e-b525-4294-89aa-a9e59390d245 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:27.096148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ecc5e-b525-4294-89aa-a9e59390d245 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:34:27.096148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ecc5e-b525-4294-89aa-a9e59390d245 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:27.096148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ecc5e-b525-4294-89aa-a9e59390d245 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:27.101726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9f7c7ba5-cc2e-4bf5-b5a6-a499ec619aa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22766b0421c645cc96078eeff2cda6b9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:27.111699 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9f7c7ba5-cc2e-4bf5-b5a6-a499ec619aa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22766b0421c645cc96078eeff2cda6b9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:27.120314 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1097/2253] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:34:27.125045 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ecc5e-b525-4294-89aa-a9e59390d245 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:27.126576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9f7c7ba5-cc2e-4bf5-b5a6-a499ec619aa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22766b0421c645cc96078eeff2cda6b9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:27.128461 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9f7c7ba5-cc2e-4bf5-b5a6-a499ec619aa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22766b0421c645cc96078eeff2cda6b9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:27.129405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9f7c7ba5-cc2e-4bf5-b5a6-a499ec619aa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22766b0421c645cc96078eeff2cda6b9'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:27.142456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ecc5e-b525-4294-89aa-a9e59390d245 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:27.148480 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794e702-0ea9-4a31-83c0-1d5c30fa9f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:27.149809 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794e702-0ea9-4a31-83c0-1d5c30fa9f59 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:27.150161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794e702-0ea9-4a31-83c0-1d5c30fa9f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:27.150491 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794e702-0ea9-4a31-83c0-1d5c30fa9f59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:27.151200 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794e702-0ea9-4a31-83c0-1d5c30fa9f59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:27.153195 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794e702-0ea9-4a31-83c0-1d5c30fa9f59 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:27.161176 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1157/2254] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:26 2022] DELETE /identity/v3/users/22766b0421c645cc96078eeff2cda6b9 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:34:27.189912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1098/2255] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:27 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:27.191099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c7ba5-cc2e-4bf5-b5a6-a499ec619aa6 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:27.192669 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9f7c7ba5-cc2e-4bf5-b5a6-a499ec619aa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:27.217780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:27.314246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c7ba5-cc2e-4bf5-b5a6-a499ec619aa6 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:27.427122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38086e2-dd39-4f9c-876c-4a2729233e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:27.428171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38086e2-dd39-4f9c-876c-4a2729233e8a admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:27.428611 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38086e2-dd39-4f9c-876c-4a2729233e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:27.430968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38086e2-dd39-4f9c-876c-4a2729233e8a admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:27.432842 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38086e2-dd39-4f9c-876c-4a2729233e8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:27.435513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38086e2-dd39-4f9c-876c-4a2729233e8a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:27.486218 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1158/2256] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:27 2022] POST /identity/v3/users => generated 292 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:27.490416 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794e702-0ea9-4a31-83c0-1d5c30fa9f59 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:27.509179 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794e702-0ea9-4a31-83c0-1d5c30fa9f59 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:27.515338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f27a4c-daf2-4dbb-8fbf-cedc5fe0dd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:27.516093 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f27a4c-daf2-4dbb-8fbf-cedc5fe0dd5e admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:27.516383 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f27a4c-daf2-4dbb-8fbf-cedc5fe0dd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:27.516851 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f27a4c-daf2-4dbb-8fbf-cedc5fe0dd5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:27.517811 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f27a4c-daf2-4dbb-8fbf-cedc5fe0dd5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:27.519213 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f27a4c-daf2-4dbb-8fbf-cedc5fe0dd5e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:27.532613 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1099/2257] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:34:27.539423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38086e2-dd39-4f9c-876c-4a2729233e8a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:27.553244 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38086e2-dd39-4f9c-876c-4a2729233e8a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:27.562767 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ce8a6-8764-4345-8d60-7c8ba73ad45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:27.563754 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ce8a6-8764-4345-8d60-7c8ba73ad45b admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:27.564740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ce8a6-8764-4345-8d60-7c8ba73ad45b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:27.564740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ce8a6-8764-4345-8d60-7c8ba73ad45b admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:27.565324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ce8a6-8764-4345-8d60-7c8ba73ad45b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:27.567983 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ce8a6-8764-4345-8d60-7c8ba73ad45b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:27.591796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1159/2258] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:27.594079 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f27a4c-daf2-4dbb-8fbf-cedc5fe0dd5e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:27.627237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f27a4c-daf2-4dbb-8fbf-cedc5fe0dd5e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:27.639040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ce8a6-8764-4345-8d60-7c8ba73ad45b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:27.646953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-117ce8a6-8764-4345-8d60-7c8ba73ad45b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:27.654207 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e30d87-bf0e-431f-b3bb-afdeb2a31ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:27.655366 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e30d87-bf0e-431f-b3bb-afdeb2a31ffd admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:27.655845 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e30d87-bf0e-431f-b3bb-afdeb2a31ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:27.656376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e30d87-bf0e-431f-b3bb-afdeb2a31ffd admin admin] PATH_INFO: `/v3/projects/3f36c1186e0f44df87d13ef836f6525b/users/448dc80e8c1b486fbdf83317209528eb/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:27.724391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e30d87-bf0e-431f-b3bb-afdeb2a31ffd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f36c1186e0f44df87d13ef836f6525b, user_id=448dc80e8c1b486fbdf83317209528eb, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:27.729481 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e30d87-bf0e-431f-b3bb-afdeb2a31ffd admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:27.730700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ce8a6-8764-4345-8d60-7c8ba73ad45b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:27.747737 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46f5fd-b017-446d-87a6-177ca610340e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:27.748460 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46f5fd-b017-446d-87a6-177ca610340e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:27.748743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46f5fd-b017-446d-87a6-177ca610340e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:27.749029 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46f5fd-b017-446d-87a6-177ca610340e admin admin] PATH_INFO: `/v3/projects/6940d4d91e2246848bb4a46716aa8e38` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:27.750238 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46f5fd-b017-446d-87a6-177ca610340e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6940d4d91e2246848bb4a46716aa8e38)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:27.752521 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46f5fd-b017-446d-87a6-177ca610340e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:27.759432 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1100/2259] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:27 2022] PUT /identity/v3/projects/3f36c1186e0f44df87d13ef836f6525b/users/448dc80e8c1b486fbdf83317209528eb/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:27.763987 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca225c9f-04a3-4050-a3b0-3bf6573c2fb5 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:27.765244 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca225c9f-04a3-4050-a3b0-3bf6573c2fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:27.766340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca225c9f-04a3-4050-a3b0-3bf6573c2fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:27.800615 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ca225c9f-04a3-4050-a3b0-3bf6573c2fb5 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:27.801972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ca225c9f-04a3-4050-a3b0-3bf6573c2fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:27.824333 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4c46f5fd-b017-446d-87a6-177ca610340e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6940d4d91e2246848bb4a46716aa8e38', 'request_id': 'req-4c46f5fd-b017-446d-87a6-177ca610340e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:27.835509 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4c46f5fd-b017-446d-87a6-177ca610340e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6940d4d91e2246848bb4a46716aa8e38', 'request_id': 'req-4c46f5fd-b017-446d-87a6-177ca610340e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:27.843177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4c46f5fd-b017-446d-87a6-177ca610340e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6940d4d91e2246848bb4a46716aa8e38', 'request_id': 'req-4c46f5fd-b017-446d-87a6-177ca610340e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:27.871498 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4c46f5fd-b017-446d-87a6-177ca610340e admin admin] The token cache is being invalidate because project 6940d4d91e2246848bb4a46716aa8e38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:27.872049 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4c46f5fd-b017-446d-87a6-177ca610340e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:27.873471 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4c46f5fd-b017-446d-87a6-177ca610340e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:27.876351 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1160/2260] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:27 2022] DELETE /identity/v3/projects/6940d4d91e2246848bb4a46716aa8e38 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:27.882835 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168d0c2-0b66-4157-afb3-6479712ceb3c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:27.883414 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168d0c2-0b66-4157-afb3-6479712ceb3c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:27.883878 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168d0c2-0b66-4157-afb3-6479712ceb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:27.913553 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1101/2261] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:27.919802 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f168d0c2-0b66-4157-afb3-6479712ceb3c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:27.921137 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f168d0c2-0b66-4157-afb3-6479712ceb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:27.931646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1161/2262] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:27.931841 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca225c9f-04a3-4050-a3b0-3bf6573c2fb5 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:27.933577 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ca225c9f-04a3-4050-a3b0-3bf6573c2fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:27.939353 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab11e8-7eb8-4313-9751-deb83fca3058 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:27.939642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab11e8-7eb8-4313-9751-deb83fca3058 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:27.939966 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab11e8-7eb8-4313-9751-deb83fca3058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:27.972489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-15ab11e8-7eb8-4313-9751-deb83fca3058 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:27.974467 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-15ab11e8-7eb8-4313-9751-deb83fca3058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:27.989477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1162/2263] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:34:27.990766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab11e8-7eb8-4313-9751-deb83fca3058 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:28.003541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab11e8-7eb8-4313-9751-deb83fca3058 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:28.010198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c6b38-e5d9-4db7-b03f-43221daac19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:28.011019 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c6b38-e5d9-4db7-b03f-43221daac19d admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:28.011301 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c6b38-e5d9-4db7-b03f-43221daac19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:28.011585 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c6b38-e5d9-4db7-b03f-43221daac19d admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:28.014638 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c6b38-e5d9-4db7-b03f-43221daac19d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:34:28.014951 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c6b38-e5d9-4db7-b03f-43221daac19d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:28.016773 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c6b38-e5d9-4db7-b03f-43221daac19d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:28.030641 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:28.041184 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1163/2264] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:34:28.044388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c6b38-e5d9-4db7-b03f-43221daac19d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:28.056079 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca225c9f-04a3-4050-a3b0-3bf6573c2fb5 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:28.073066 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c6b38-e5d9-4db7-b03f-43221daac19d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:28.073310 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02011bae-a5a2-4ec6-89b8-239170c5ee26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:28.073310 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02011bae-a5a2-4ec6-89b8-239170c5ee26 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:28.073310 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02011bae-a5a2-4ec6-89b8-239170c5ee26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:28.073310 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02011bae-a5a2-4ec6-89b8-239170c5ee26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:28.073614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0770ebc-840a-449c-bfbc-13c523cbd9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:28.073614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0770ebc-840a-449c-bfbc-13c523cbd9e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:28.073614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0770ebc-840a-449c-bfbc-13c523cbd9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:28.073614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0770ebc-840a-449c-bfbc-13c523cbd9e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:28.074604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0770ebc-840a-449c-bfbc-13c523cbd9e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:28.080648 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0770ebc-840a-449c-bfbc-13c523cbd9e3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:28.084524 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02011bae-a5a2-4ec6-89b8-239170c5ee26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:28.086135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02011bae-a5a2-4ec6-89b8-239170c5ee26 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:28.097603 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1102/2265] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:34:28.138236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1164/2266] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:28 2022] POST /identity/v3/projects => generated 399 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:34:28.139378 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02011bae-a5a2-4ec6-89b8-239170c5ee26 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:28.154427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02011bae-a5a2-4ec6-89b8-239170c5ee26 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:28.189394 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:28.263283 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b4571-8a8a-47b9-8f0f-29bfab849f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:28.264187 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b4571-8a8a-47b9-8f0f-29bfab849f1a admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:28.264539 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b4571-8a8a-47b9-8f0f-29bfab849f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:28.264878 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b4571-8a8a-47b9-8f0f-29bfab849f1a admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:28.265595 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b4571-8a8a-47b9-8f0f-29bfab849f1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:28.268244 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b4571-8a8a-47b9-8f0f-29bfab849f1a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:28.311335 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1103/2267] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:28 2022] POST /identity/v3/users => generated 300 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:28.315558 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0770ebc-840a-449c-bfbc-13c523cbd9e3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:28.327036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0770ebc-840a-449c-bfbc-13c523cbd9e3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:28.332720 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97067654-5e2a-4e94-8eb1-5d6f50238570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:28.333488 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97067654-5e2a-4e94-8eb1-5d6f50238570 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:28.333777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97067654-5e2a-4e94-8eb1-5d6f50238570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:28.334041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97067654-5e2a-4e94-8eb1-5d6f50238570 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:28.334787 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97067654-5e2a-4e94-8eb1-5d6f50238570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:28.337022 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97067654-5e2a-4e94-8eb1-5d6f50238570 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:28.355036 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b4571-8a8a-47b9-8f0f-29bfab849f1a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:28.355457 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1165/2268] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:28.375723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b4571-8a8a-47b9-8f0f-29bfab849f1a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:28.381477 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde71309-5ebc-4515-b6bf-c12a05ff869d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:28.382067 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde71309-5ebc-4515-b6bf-c12a05ff869d admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:28.389955 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde71309-5ebc-4515-b6bf-c12a05ff869d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:28.390297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde71309-5ebc-4515-b6bf-c12a05ff869d admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:28.391110 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde71309-5ebc-4515-b6bf-c12a05ff869d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:28.395563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde71309-5ebc-4515-b6bf-c12a05ff869d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:28.411898 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1104/2269] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:28.424156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97067654-5e2a-4e94-8eb1-5d6f50238570 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:28.440994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97067654-5e2a-4e94-8eb1-5d6f50238570 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:28.449151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c302b-5e59-4110-8367-7adb5bf301cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:28.450020 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c302b-5e59-4110-8367-7adb5bf301cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:28.450463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c302b-5e59-4110-8367-7adb5bf301cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:28.450882 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c302b-5e59-4110-8367-7adb5bf301cc admin admin] PATH_INFO: `/v3/projects/7b1586a95edd4b04943d104f9f814757/users/e7c0f74cbc364b12989f1340fbe6d942/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:28.478841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c302b-5e59-4110-8367-7adb5bf301cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b1586a95edd4b04943d104f9f814757, user_id=e7c0f74cbc364b12989f1340fbe6d942, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:28.482390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c302b-5e59-4110-8367-7adb5bf301cc admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:28.523280 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1166/2270] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:28 2022] PUT /identity/v3/projects/7b1586a95edd4b04943d104f9f814757/users/e7c0f74cbc364b12989f1340fbe6d942/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:28.529924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c09be-b235-48bc-82ef-4c7c2e657f90 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:28.530257 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c09be-b235-48bc-82ef-4c7c2e657f90 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:28.531395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c09be-b235-48bc-82ef-4c7c2e657f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:28.580463 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-7d3c09be-b235-48bc-82ef-4c7c2e657f90 None None] MFA Rules not processed for user `e7c0f74cbc364b12989f1340fbe6d942`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:28.582269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7d3c09be-b235-48bc-82ef-4c7c2e657f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:28.764521 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1105/2271] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:28 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:28.768332 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c302b-5e59-4110-8367-7adb5bf301cc admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:28.907779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c302b-5e59-4110-8367-7adb5bf301cc admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:28.913422 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96ac4d-36f8-4211-bfda-af1077568506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:28.914320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96ac4d-36f8-4211-bfda-af1077568506 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:28.915263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96ac4d-36f8-4211-bfda-af1077568506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:28.915741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96ac4d-36f8-4211-bfda-af1077568506 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:28.916644 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96ac4d-36f8-4211-bfda-af1077568506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:28.919525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96ac4d-36f8-4211-bfda-af1077568506 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:29.022568 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1167/2272] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:28 2022] POST /identity/v3/projects => generated 401 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:34:29.027512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c09be-b235-48bc-82ef-4c7c2e657f90 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:29.066445 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c09be-b235-48bc-82ef-4c7c2e657f90 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:29.288440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f65fea-5ad9-413a-b3e5-d91ece29f93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:29.292819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f65fea-5ad9-413a-b3e5-d91ece29f93d admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:29.293100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f65fea-5ad9-413a-b3e5-d91ece29f93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:29.293379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f65fea-5ad9-413a-b3e5-d91ece29f93d admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:29.296350 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f65fea-5ad9-413a-b3e5-d91ece29f93d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:29.314459 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f65fea-5ad9-413a-b3e5-d91ece29f93d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:29.367726 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1106/2273] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:29 2022] POST /identity/v3/users => generated 307 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:29.373136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96ac4d-36f8-4211-bfda-af1077568506 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:29.390211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96ac4d-36f8-4211-bfda-af1077568506 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:29.404951 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23b824-0fa8-452b-913f-87e95a87e210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:29.405857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23b824-0fa8-452b-913f-87e95a87e210 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:29.406352 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23b824-0fa8-452b-913f-87e95a87e210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:29.406804 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23b824-0fa8-452b-913f-87e95a87e210 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:29.409842 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23b824-0fa8-452b-913f-87e95a87e210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:29.412204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23b824-0fa8-452b-913f-87e95a87e210 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:29.459261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1168/2274] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:29.463485 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f65fea-5ad9-413a-b3e5-d91ece29f93d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:29.481260 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f65fea-5ad9-413a-b3e5-d91ece29f93d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:29.509376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98006051-7b15-4201-be53-81e6a2163f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:29.510056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98006051-7b15-4201-be53-81e6a2163f0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:29.510327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98006051-7b15-4201-be53-81e6a2163f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:29.516171 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98006051-7b15-4201-be53-81e6a2163f0a admin admin] PATH_INFO: `/v3/projects/7448a1301ade4188811365e4b7e580a0/users/a8036dbc918747b2898113836a301421/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:29.566445 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98006051-7b15-4201-be53-81e6a2163f0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7448a1301ade4188811365e4b7e580a0, user_id=a8036dbc918747b2898113836a301421, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:29.570979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98006051-7b15-4201-be53-81e6a2163f0a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:29.581524 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:29.606767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1107/2275] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:29 2022] PUT /identity/v3/projects/7448a1301ade4188811365e4b7e580a0/users/a8036dbc918747b2898113836a301421/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:29.612081 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23b824-0fa8-452b-913f-87e95a87e210 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:29.657739 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23b824-0fa8-452b-913f-87e95a87e210 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:29.896746 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb4a15-939b-43dd-81fd-0e104035bf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:29.897821 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb4a15-939b-43dd-81fd-0e104035bf71 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:29.899253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb4a15-939b-43dd-81fd-0e104035bf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:29.900215 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb4a15-939b-43dd-81fd-0e104035bf71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:29.901210 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb4a15-939b-43dd-81fd-0e104035bf71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:29.904089 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb4a15-939b-43dd-81fd-0e104035bf71 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:29.928925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1169/2276] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:29.940553 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98006051-7b15-4201-be53-81e6a2163f0a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:29.971144 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98006051-7b15-4201-be53-81e6a2163f0a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:29.977453 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21313b4d-7a1b-4080-b6c4-3fe2746e78f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:29.978793 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21313b4d-7a1b-4080-b6c4-3fe2746e78f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:29.979385 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21313b4d-7a1b-4080-b6c4-3fe2746e78f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:29.979873 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21313b4d-7a1b-4080-b6c4-3fe2746e78f4 admin admin] PATH_INFO: `/v3/projects/7448a1301ade4188811365e4b7e580a0/users/a8036dbc918747b2898113836a301421/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:29.986074 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21313b4d-7a1b-4080-b6c4-3fe2746e78f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7448a1301ade4188811365e4b7e580a0, user_id=a8036dbc918747b2898113836a301421, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:29.991058 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21313b4d-7a1b-4080-b6c4-3fe2746e78f4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:30.011987 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1108/2277] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:29 2022] PUT /identity/v3/projects/7448a1301ade4188811365e4b7e580a0/users/a8036dbc918747b2898113836a301421/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:30.021269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdd1b4-a7a6-47c1-a6fd-c4eb18833ce6 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:30.021540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdd1b4-a7a6-47c1-a6fd-c4eb18833ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:30.021805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdd1b4-a7a6-47c1-a6fd-c4eb18833ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:30.049173 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:30.053984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-58fdd1b4-a7a6-47c1-a6fd-c4eb18833ce6 None None] MFA Rules not processed for user `a8036dbc918747b2898113836a301421`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:30.056100 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-58fdd1b4-a7a6-47c1-a6fd-c4eb18833ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:30.160573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1170/2278] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:30 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:30.170649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21313b4d-7a1b-4080-b6c4-3fe2746e78f4 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:30.173447 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-21313b4d-7a1b-4080-b6c4-3fe2746e78f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:30.298297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21313b4d-7a1b-4080-b6c4-3fe2746e78f4 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:30.310280 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf692b-e66b-4fc2-bc29-e82f61fe64e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:30.312590 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf692b-e66b-4fc2-bc29-e82f61fe64e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:30.313451 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf692b-e66b-4fc2-bc29-e82f61fe64e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:30.313900 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf692b-e66b-4fc2-bc29-e82f61fe64e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:30.329766 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf692b-e66b-4fc2-bc29-e82f61fe64e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:30.332788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf692b-e66b-4fc2-bc29-e82f61fe64e8 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:30.349683 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1109/2279] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:30 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:30.480233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdd1b4-a7a6-47c1-a6fd-c4eb18833ce6 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:30.483647 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-58fdd1b4-a7a6-47c1-a6fd-c4eb18833ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:30.500035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdd1b4-a7a6-47c1-a6fd-c4eb18833ce6 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:30.516843 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd22375-6eb3-4a48-b0b4-13431fc8a822 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:30.517740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd22375-6eb3-4a48-b0b4-13431fc8a822 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:30.518177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd22375-6eb3-4a48-b0b4-13431fc8a822 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:30.518554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd22375-6eb3-4a48-b0b4-13431fc8a822 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:30.538980 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf692b-e66b-4fc2-bc29-e82f61fe64e8 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:30.541453 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-66bf692b-e66b-4fc2-bc29-e82f61fe64e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:30.598661 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd22375-6eb3-4a48-b0b4-13431fc8a822 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:30.600328 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd22375-6eb3-4a48-b0b4-13431fc8a822 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:30.611981 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1171/2280] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:30 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:30.657355 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf692b-e66b-4fc2-bc29-e82f61fe64e8 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:30.668674 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4f20d-c1fe-4d5f-bb19-dfb7ae9818d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:30.670693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f20d-c1fe-4d5f-bb19-dfb7ae9818d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:30.671339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f20d-c1fe-4d5f-bb19-dfb7ae9818d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:30.672048 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4f20d-c1fe-4d5f-bb19-dfb7ae9818d4 admin admin] PATH_INFO: `/v3/users/3c6b2b6efe4a4173ab75101785407495` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:30.674035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4f20d-c1fe-4d5f-bb19-dfb7ae9818d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c6b2b6efe4a4173ab75101785407495)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:30.681003 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4f20d-c1fe-4d5f-bb19-dfb7ae9818d4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:30.834795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ecd4f20d-c1fe-4d5f-bb19-dfb7ae9818d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c6b2b6efe4a4173ab75101785407495'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:30.859717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ecd4f20d-c1fe-4d5f-bb19-dfb7ae9818d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c6b2b6efe4a4173ab75101785407495'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:30.871681 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ecd4f20d-c1fe-4d5f-bb19-dfb7ae9818d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c6b2b6efe4a4173ab75101785407495'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:30.873527 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ecd4f20d-c1fe-4d5f-bb19-dfb7ae9818d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c6b2b6efe4a4173ab75101785407495'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:30.893521 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ecd4f20d-c1fe-4d5f-bb19-dfb7ae9818d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c6b2b6efe4a4173ab75101785407495'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:30.904068 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1110/2281] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:30 2022] DELETE /identity/v3/users/3c6b2b6efe4a4173ab75101785407495 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:34:30.909333 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46023f9e-ad6b-49db-b7e9-cdfa9ad85288 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:30.909895 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46023f9e-ad6b-49db-b7e9-cdfa9ad85288 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:30.909895 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46023f9e-ad6b-49db-b7e9-cdfa9ad85288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:30.936155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-46023f9e-ad6b-49db-b7e9-cdfa9ad85288 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:30.937190 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-46023f9e-ad6b-49db-b7e9-cdfa9ad85288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:30.943992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8621a3-fabb-47bb-9923-533c42023c6d None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:30.944741 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8621a3-fabb-47bb-9923-533c42023c6d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:30.945348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8621a3-fabb-47bb-9923-533c42023c6d None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:30.950100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1111/2282] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 29 20:34:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:34:30.987710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a37abf-b7f6-4e18-ab25-a5c1040222eb None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:30.988229 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a37abf-b7f6-4e18-ab25-a5c1040222eb None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:30.988668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a37abf-b7f6-4e18-ab25-a5c1040222eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:31.013669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:31.065822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-71a37abf-b7f6-4e18-ab25-a5c1040222eb None None] MFA Rules not processed for user `fa0e659eee5e4a9da8d3d7337cb73798`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:31.070792 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-71a37abf-b7f6-4e18-ab25-a5c1040222eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:31.176217 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1172/2283] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:31.189302 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46023f9e-ad6b-49db-b7e9-cdfa9ad85288 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:31.193328 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-46023f9e-ad6b-49db-b7e9-cdfa9ad85288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:31.223854 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46023f9e-ad6b-49db-b7e9-cdfa9ad85288 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:31.239423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27752606-17d5-498f-a122-8ec9fbc569ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:31.240373 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27752606-17d5-498f-a122-8ec9fbc569ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:31.242337 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27752606-17d5-498f-a122-8ec9fbc569ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:31.242855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27752606-17d5-498f-a122-8ec9fbc569ed admin admin] PATH_INFO: `/v3/users/03117ffa81334df8a3467adbfaf96059` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:31.247724 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27752606-17d5-498f-a122-8ec9fbc569ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=03117ffa81334df8a3467adbfaf96059)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:31.253542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27752606-17d5-498f-a122-8ec9fbc569ed admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:31.310933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1112/2284] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 29 20:34:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:31.313296 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a37abf-b7f6-4e18-ab25-a5c1040222eb None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:31.316673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-71a37abf-b7f6-4e18-ab25-a5c1040222eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:31.345285 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:31.385691 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-27752606-17d5-498f-a122-8ec9fbc569ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03117ffa81334df8a3467adbfaf96059'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:31.414155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-27752606-17d5-498f-a122-8ec9fbc569ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03117ffa81334df8a3467adbfaf96059'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:31.478247 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-27752606-17d5-498f-a122-8ec9fbc569ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03117ffa81334df8a3467adbfaf96059'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:31.479863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-27752606-17d5-498f-a122-8ec9fbc569ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03117ffa81334df8a3467adbfaf96059'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:31.482209 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-27752606-17d5-498f-a122-8ec9fbc569ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03117ffa81334df8a3467adbfaf96059'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:31.501758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a37abf-b7f6-4e18-ab25-a5c1040222eb None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:31.520028 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd3309-3028-4547-9f97-c7243263aed1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:31.520669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd3309-3028-4547-9f97-c7243263aed1 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:31.520947 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd3309-3028-4547-9f97-c7243263aed1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:31.521222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd3309-3028-4547-9f97-c7243263aed1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:31.526295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-13dd3309-3028-4547-9f97-c7243263aed1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:31.532106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1173/2285] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:31 2022] DELETE /identity/v3/users/03117ffa81334df8a3467adbfaf96059 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:31.533257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27752606-17d5-498f-a122-8ec9fbc569ed admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:31.540399 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-27752606-17d5-498f-a122-8ec9fbc569ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:31.573196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd3309-3028-4547-9f97-c7243263aed1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:31.575240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd3309-3028-4547-9f97-c7243263aed1 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:31.596339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1113/2286] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:31.779425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27752606-17d5-498f-a122-8ec9fbc569ed admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:31.789545 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de799cb-232c-4e64-8f8f-39224f8fc15a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0e659eee5e4a9da8d3d7337cb73798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:31.791445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de799cb-232c-4e64-8f8f-39224f8fc15a service placement] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:31.791782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de799cb-232c-4e64-8f8f-39224f8fc15a service placement] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:31.792155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de799cb-232c-4e64-8f8f-39224f8fc15a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:31.795104 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1de799cb-232c-4e64-8f8f-39224f8fc15a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:31.837866 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de799cb-232c-4e64-8f8f-39224f8fc15a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:31.842780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de799cb-232c-4e64-8f8f-39224f8fc15a service placement] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:31.870274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1174/2287] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:32.116053 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd3309-3028-4547-9f97-c7243263aed1 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:32.120514 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-13dd3309-3028-4547-9f97-c7243263aed1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:32.155438 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd3309-3028-4547-9f97-c7243263aed1 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:32.293487 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de799cb-232c-4e64-8f8f-39224f8fc15a service placement] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:32.295819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1de799cb-232c-4e64-8f8f-39224f8fc15a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:32.393684 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf3dfa-bae1-496c-a339-a2f859d740ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:32.399915 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf3dfa-bae1-496c-a339-a2f859d740ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:32.400474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf3dfa-bae1-496c-a339-a2f859d740ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:32.400952 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf3dfa-bae1-496c-a339-a2f859d740ea admin admin] PATH_INFO: `/v3/projects/6314d8436170404097713aa7764dff94` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:32.403048 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf3dfa-bae1-496c-a339-a2f859d740ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=6314d8436170404097713aa7764dff94)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:32.405185 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cf3dfa-bae1-496c-a339-a2f859d740ea admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:32.513953 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de799cb-232c-4e64-8f8f-39224f8fc15a service placement] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:32.546270 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:32.552381 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e7cf3dfa-bae1-496c-a339-a2f859d740ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6314d8436170404097713aa7764dff94', 'request_id': 'req-e7cf3dfa-bae1-496c-a339-a2f859d740ea'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:32.572089 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e7cf3dfa-bae1-496c-a339-a2f859d740ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6314d8436170404097713aa7764dff94', 'request_id': 'req-e7cf3dfa-bae1-496c-a339-a2f859d740ea'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:32.590198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e7cf3dfa-bae1-496c-a339-a2f859d740ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6314d8436170404097713aa7764dff94', 'request_id': 'req-e7cf3dfa-bae1-496c-a339-a2f859d740ea'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:32.621887 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e7cf3dfa-bae1-496c-a339-a2f859d740ea admin admin] The token cache is being invalidate because project 6314d8436170404097713aa7764dff94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:32.623510 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e7cf3dfa-bae1-496c-a339-a2f859d740ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:32.625484 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e7cf3dfa-bae1-496c-a339-a2f859d740ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:32.628991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1114/2288] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:32 2022] DELETE /identity/v3/projects/6314d8436170404097713aa7764dff94 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:32.637753 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a91bb6-3ef2-4c01-88d5-79e602da472c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:32.639494 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a91bb6-3ef2-4c01-88d5-79e602da472c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:32.640056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a91bb6-3ef2-4c01-88d5-79e602da472c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:32.654376 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:32.686786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99a1de-ccd5-4e6a-8b57-39148c6d3f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:32.688284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99a1de-ccd5-4e6a-8b57-39148c6d3f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:32.689285 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99a1de-ccd5-4e6a-8b57-39148c6d3f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:32.689903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99a1de-ccd5-4e6a-8b57-39148c6d3f54 admin admin] PATH_INFO: `/v3/projects/6bbbffea79eb40389f9f3b06cf7fb9a3` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:32.698232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99a1de-ccd5-4e6a-8b57-39148c6d3f54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bbbffea79eb40389f9f3b06cf7fb9a3)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:32.704541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99a1de-ccd5-4e6a-8b57-39148c6d3f54 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:32.708551 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-60a91bb6-3ef2-4c01-88d5-79e602da472c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:32.709850 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-60a91bb6-3ef2-4c01-88d5-79e602da472c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:32.740005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1115/2289] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:32.749183 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60247b2-ed42-4071-bffc-c0139295576b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:32.750016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60247b2-ed42-4071-bffc-c0139295576b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:32.750740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60247b2-ed42-4071-bffc-c0139295576b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:32.790009 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-a60247b2-ed42-4071-bffc-c0139295576b None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:32.792733 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a60247b2-ed42-4071-bffc-c0139295576b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:32.818282 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1116/2290] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:32.819683 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60247b2-ed42-4071-bffc-c0139295576b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:32.850107 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60247b2-ed42-4071-bffc-c0139295576b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:32.892895 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7b99a1de-ccd5-4e6a-8b57-39148c6d3f54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bbbffea79eb40389f9f3b06cf7fb9a3', 'request_id': 'req-7b99a1de-ccd5-4e6a-8b57-39148c6d3f54'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:32.908492 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7b99a1de-ccd5-4e6a-8b57-39148c6d3f54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bbbffea79eb40389f9f3b06cf7fb9a3', 'request_id': 'req-7b99a1de-ccd5-4e6a-8b57-39148c6d3f54'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:32.919590 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7b99a1de-ccd5-4e6a-8b57-39148c6d3f54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bbbffea79eb40389f9f3b06cf7fb9a3', 'request_id': 'req-7b99a1de-ccd5-4e6a-8b57-39148c6d3f54'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:32.955404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7b99a1de-ccd5-4e6a-8b57-39148c6d3f54 admin admin] The token cache is being invalidate because project 6bbbffea79eb40389f9f3b06cf7fb9a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:32.956814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7b99a1de-ccd5-4e6a-8b57-39148c6d3f54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:32.957424 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7b99a1de-ccd5-4e6a-8b57-39148c6d3f54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:32.960455 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1175/2291] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:32 2022] DELETE /identity/v3/projects/6bbbffea79eb40389f9f3b06cf7fb9a3 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:32.965784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99a1de-ccd5-4e6a-8b57-39148c6d3f54 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:32.967448 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7b99a1de-ccd5-4e6a-8b57-39148c6d3f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:33.051569 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a61346-c713-4d12-b4e3-552ad6bbc5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:33.052232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a61346-c713-4d12-b4e3-552ad6bbc5e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:33.052583 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a61346-c713-4d12-b4e3-552ad6bbc5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:33.052771 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a61346-c713-4d12-b4e3-552ad6bbc5e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:33.053585 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a61346-c713-4d12-b4e3-552ad6bbc5e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:34:33.053886 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a61346-c713-4d12-b4e3-552ad6bbc5e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:33.056348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a61346-c713-4d12-b4e3-552ad6bbc5e2 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:33.083501 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1117/2292] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:34:33.085859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a61346-c713-4d12-b4e3-552ad6bbc5e2 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:33.089252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e0a61346-c713-4d12-b4e3-552ad6bbc5e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:33.119982 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99a1de-ccd5-4e6a-8b57-39148c6d3f54 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:33.124062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a61346-c713-4d12-b4e3-552ad6bbc5e2 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:33.132548 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba79e07d-b298-4d89-a484-4eff23b3bc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:33.133214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79e07d-b298-4d89-a484-4eff23b3bc7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:33.133532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79e07d-b298-4d89-a484-4eff23b3bc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:33.134394 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79e07d-b298-4d89-a484-4eff23b3bc7a admin admin] PATH_INFO: `/v3/users/78442da1a497473d8c3d773c1e624902` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:33.136017 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba79e07d-b298-4d89-a484-4eff23b3bc7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=78442da1a497473d8c3d773c1e624902)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:33.137942 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba79e07d-b298-4d89-a484-4eff23b3bc7a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:33.141725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebde250-f680-477c-9912-c3667ec01cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:33.142422 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebde250-f680-477c-9912-c3667ec01cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:33.142720 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebde250-f680-477c-9912-c3667ec01cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:33.142990 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebde250-f680-477c-9912-c3667ec01cb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:33.143625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebde250-f680-477c-9912-c3667ec01cb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:33.145681 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebde250-f680-477c-9912-c3667ec01cb7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:33.206595 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1118/2293] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:33 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:33.208393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebde250-f680-477c-9912-c3667ec01cb7 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:33.273748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ba79e07d-b298-4d89-a484-4eff23b3bc7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78442da1a497473d8c3d773c1e624902'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:33.328637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ba79e07d-b298-4d89-a484-4eff23b3bc7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78442da1a497473d8c3d773c1e624902'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:33.351728 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ba79e07d-b298-4d89-a484-4eff23b3bc7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78442da1a497473d8c3d773c1e624902'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:33.353297 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ba79e07d-b298-4d89-a484-4eff23b3bc7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78442da1a497473d8c3d773c1e624902'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:33.354262 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ba79e07d-b298-4d89-a484-4eff23b3bc7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78442da1a497473d8c3d773c1e624902'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:33.368783 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebde250-f680-477c-9912-c3667ec01cb7 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:33.374046 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:33.400419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa24b37-d7f3-44e0-a376-91cfb41e7045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:33.401743 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa24b37-d7f3-44e0-a376-91cfb41e7045 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:33.402018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa24b37-d7f3-44e0-a376-91cfb41e7045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:33.402285 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa24b37-d7f3-44e0-a376-91cfb41e7045 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:33.402923 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa24b37-d7f3-44e0-a376-91cfb41e7045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:33.409531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1176/2294] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:32 2022] DELETE /identity/v3/users/78442da1a497473d8c3d773c1e624902 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:33.409859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa24b37-d7f3-44e0-a376-91cfb41e7045 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:33.462121 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1119/2295] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:33 2022] POST /identity/v3/users => generated 294 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:33.463914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba79e07d-b298-4d89-a484-4eff23b3bc7a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:33.466215 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ba79e07d-b298-4d89-a484-4eff23b3bc7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:33.595767 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba79e07d-b298-4d89-a484-4eff23b3bc7a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:33.602468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc7f0d-4e7d-412c-aee7-ea0fba06bed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:33.603697 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc7f0d-4e7d-412c-aee7-ea0fba06bed3 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:33.604601 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc7f0d-4e7d-412c-aee7-ea0fba06bed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:33.604601 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc7f0d-4e7d-412c-aee7-ea0fba06bed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:33.604986 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc7f0d-4e7d-412c-aee7-ea0fba06bed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:33.606693 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc7f0d-4e7d-412c-aee7-ea0fba06bed3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:33.621943 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1177/2296] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:33.626516 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa24b37-d7f3-44e0-a376-91cfb41e7045 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:33.643528 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa24b37-d7f3-44e0-a376-91cfb41e7045 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:33.649708 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e0ef6-e87d-4728-bb6d-3a3377d558be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:33.650345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e0ef6-e87d-4728-bb6d-3a3377d558be admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:33.650627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e0ef6-e87d-4728-bb6d-3a3377d558be admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:33.650906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e0ef6-e87d-4728-bb6d-3a3377d558be admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:33.651599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e0ef6-e87d-4728-bb6d-3a3377d558be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:33.653525 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e0ef6-e87d-4728-bb6d-3a3377d558be admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:33.671606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1120/2297] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:33.673397 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc7f0d-4e7d-412c-aee7-ea0fba06bed3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:33.690915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc7f0d-4e7d-412c-aee7-ea0fba06bed3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:33.698069 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c61a2-3df8-4d22-865b-6a9b7a37c1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:33.699094 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c61a2-3df8-4d22-865b-6a9b7a37c1e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:33.708321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c61a2-3df8-4d22-865b-6a9b7a37c1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:33.708923 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c61a2-3df8-4d22-865b-6a9b7a37c1e3 admin admin] PATH_INFO: `/v3/projects/bf00868e84384ee59320184a5e1c46c8/users/068f28eb74e74650a2b80d29664109ae/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:33.733232 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c61a2-3df8-4d22-865b-6a9b7a37c1e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf00868e84384ee59320184a5e1c46c8, user_id=068f28eb74e74650a2b80d29664109ae, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:33.736486 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c61a2-3df8-4d22-865b-6a9b7a37c1e3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:33.758813 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1178/2298] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:33 2022] PUT /identity/v3/projects/bf00868e84384ee59320184a5e1c46c8/users/068f28eb74e74650a2b80d29664109ae/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:33.765358 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3207af-fdc7-4a81-a27e-857c10e5d5be None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:33.765762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3207af-fdc7-4a81-a27e-857c10e5d5be None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:33.766220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3207af-fdc7-4a81-a27e-857c10e5d5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:33.797069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-fd3207af-fdc7-4a81-a27e-857c10e5d5be None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:33.798354 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-fd3207af-fdc7-4a81-a27e-857c10e5d5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:33.806364 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:33.920463 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1121/2299] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:33.933186 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c61a2-3df8-4d22-865b-6a9b7a37c1e3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:33.937809 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-142c61a2-3df8-4d22-865b-6a9b7a37c1e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:33.992811 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3207af-fdc7-4a81-a27e-857c10e5d5be None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:33.995146 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-fd3207af-fdc7-4a81-a27e-857c10e5d5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:34.018770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3207af-fdc7-4a81-a27e-857c10e5d5be None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:34.034981 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9154db5-745e-40ee-a557-53717b3d9b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:34.035771 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9154db5-745e-40ee-a557-53717b3d9b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:34.036446 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9154db5-745e-40ee-a557-53717b3d9b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:34.036574 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9154db5-745e-40ee-a557-53717b3d9b41 admin admin] PATH_INFO: `/v3/projects/7a720ffcd10b41edb9d30a0bd697653f` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:34.037768 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9154db5-745e-40ee-a557-53717b3d9b41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a720ffcd10b41edb9d30a0bd697653f)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:34.039726 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9154db5-745e-40ee-a557-53717b3d9b41 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:34.051924 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c61a2-3df8-4d22-865b-6a9b7a37c1e3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:34.064500 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d24899-b29d-4419-abb4-835646b6cf43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:34.065083 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d24899-b29d-4419-abb4-835646b6cf43 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:34.065371 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d24899-b29d-4419-abb4-835646b6cf43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:34.065648 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d24899-b29d-4419-abb4-835646b6cf43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:34.079780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d24899-b29d-4419-abb4-835646b6cf43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:34.081930 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d24899-b29d-4419-abb4-835646b6cf43 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:34.109546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1179/2300] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:34.111301 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d24899-b29d-4419-abb4-835646b6cf43 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:34.113677 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e1d24899-b29d-4419-abb4-835646b6cf43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:34.132735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d24899-b29d-4419-abb4-835646b6cf43 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:34.155103 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d9154db5-745e-40ee-a557-53717b3d9b41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a720ffcd10b41edb9d30a0bd697653f', 'request_id': 'req-d9154db5-745e-40ee-a557-53717b3d9b41'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:34.171751 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d9154db5-745e-40ee-a557-53717b3d9b41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a720ffcd10b41edb9d30a0bd697653f', 'request_id': 'req-d9154db5-745e-40ee-a557-53717b3d9b41'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:34.184545 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d9154db5-745e-40ee-a557-53717b3d9b41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a720ffcd10b41edb9d30a0bd697653f', 'request_id': 'req-d9154db5-745e-40ee-a557-53717b3d9b41'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:34.209989 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ca712-3e5f-454d-9de1-5419a80028f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:34.210737 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ca712-3e5f-454d-9de1-5419a80028f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:34.211025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ca712-3e5f-454d-9de1-5419a80028f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:34.211358 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ca712-3e5f-454d-9de1-5419a80028f8 admin admin] PATH_INFO: `/v3/users/12ee7183e3064ca184156234296594fe` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:34.211692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d9154db5-745e-40ee-a557-53717b3d9b41 admin admin] The token cache is being invalidate because project 7a720ffcd10b41edb9d30a0bd697653f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:34.212020 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d9154db5-745e-40ee-a557-53717b3d9b41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:34.212777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ca712-3e5f-454d-9de1-5419a80028f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12ee7183e3064ca184156234296594fe)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:34.213337 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d9154db5-745e-40ee-a557-53717b3d9b41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:34.214876 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ca712-3e5f-454d-9de1-5419a80028f8 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:34.216058 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1122/2301] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:33 2022] DELETE /identity/v3/projects/7a720ffcd10b41edb9d30a0bd697653f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:34.219866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9154db5-745e-40ee-a557-53717b3d9b41 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:34.222027 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d9154db5-745e-40ee-a557-53717b3d9b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:34.235904 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:34.253669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9154db5-745e-40ee-a557-53717b3d9b41 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:34.260969 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f7804-90e1-426c-9e0b-69840168ce1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:34.261719 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f7804-90e1-426c-9e0b-69840168ce1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:34.262002 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f7804-90e1-426c-9e0b-69840168ce1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:34.262309 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f7804-90e1-426c-9e0b-69840168ce1c admin admin] PATH_INFO: `/v3/users/d08a68e5233e4969b591a313763bd225` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:34.263687 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f7804-90e1-426c-9e0b-69840168ce1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d08a68e5233e4969b591a313763bd225)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:34.265999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f7804-90e1-426c-9e0b-69840168ce1c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:34.320109 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6a6ca712-3e5f-454d-9de1-5419a80028f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12ee7183e3064ca184156234296594fe'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:34.345889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6a6ca712-3e5f-454d-9de1-5419a80028f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12ee7183e3064ca184156234296594fe'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:34.356468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6a6ca712-3e5f-454d-9de1-5419a80028f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12ee7183e3064ca184156234296594fe'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:34.357461 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6a6ca712-3e5f-454d-9de1-5419a80028f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12ee7183e3064ca184156234296594fe'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:34.358234 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6a6ca712-3e5f-454d-9de1-5419a80028f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12ee7183e3064ca184156234296594fe'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:34.399241 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1180/2302] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:34 2022] DELETE /identity/v3/users/12ee7183e3064ca184156234296594fe => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:34.399908 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3f855-48d1-4a74-8f23-81f4aa213d4e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:34.400388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3f855-48d1-4a74-8f23-81f4aa213d4e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:34.401935 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3f855-48d1-4a74-8f23-81f4aa213d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:34.408933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f67f7804-90e1-426c-9e0b-69840168ce1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd08a68e5233e4969b591a313763bd225'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:34.431766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-51e3f855-48d1-4a74-8f23-81f4aa213d4e None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:34.434014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-51e3f855-48d1-4a74-8f23-81f4aa213d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:34.434481 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f67f7804-90e1-426c-9e0b-69840168ce1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd08a68e5233e4969b591a313763bd225'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:34.448635 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f67f7804-90e1-426c-9e0b-69840168ce1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd08a68e5233e4969b591a313763bd225'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:34.449900 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f67f7804-90e1-426c-9e0b-69840168ce1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd08a68e5233e4969b591a313763bd225'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:34.470617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f67f7804-90e1-426c-9e0b-69840168ce1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd08a68e5233e4969b591a313763bd225'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:34.487960 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1123/2303] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:34 2022] DELETE /identity/v3/users/d08a68e5233e4969b591a313763bd225 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:34.565577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1181/2304] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:34.607400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f7804-90e1-426c-9e0b-69840168ce1c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:34.612243 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f67f7804-90e1-426c-9e0b-69840168ce1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:34.626934 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:34.765452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f7804-90e1-426c-9e0b-69840168ce1c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:34.783109 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ff2e3-666f-4a84-a1da-9ee7d66ecdf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:34.783748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ff2e3-666f-4a84-a1da-9ee7d66ecdf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:34.784095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ff2e3-666f-4a84-a1da-9ee7d66ecdf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:34.784370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ff2e3-666f-4a84-a1da-9ee7d66ecdf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:34.895927 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ff2e3-666f-4a84-a1da-9ee7d66ecdf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:34.897456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ff2e3-666f-4a84-a1da-9ee7d66ecdf5 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:34.911587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1124/2305] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:35.523379 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3f855-48d1-4a74-8f23-81f4aa213d4e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:35.525694 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-51e3f855-48d1-4a74-8f23-81f4aa213d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:35.538750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3f855-48d1-4a74-8f23-81f4aa213d4e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:35.545389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b6fd8-dc65-46b3-966d-b5db6e58abc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:35.546398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b6fd8-dc65-46b3-966d-b5db6e58abc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:35.546894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b6fd8-dc65-46b3-966d-b5db6e58abc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:35.547377 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b6fd8-dc65-46b3-966d-b5db6e58abc3 admin admin] PATH_INFO: `/v3/projects/4f742cc32a40422ea34d5959731f4444` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:35.548962 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b6fd8-dc65-46b3-966d-b5db6e58abc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f742cc32a40422ea34d5959731f4444)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:35.552419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5b6fd8-dc65-46b3-966d-b5db6e58abc3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:35.619233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3f5b6fd8-dc65-46b3-966d-b5db6e58abc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f742cc32a40422ea34d5959731f4444', 'request_id': 'req-3f5b6fd8-dc65-46b3-966d-b5db6e58abc3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:35.632065 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3f5b6fd8-dc65-46b3-966d-b5db6e58abc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f742cc32a40422ea34d5959731f4444', 'request_id': 'req-3f5b6fd8-dc65-46b3-966d-b5db6e58abc3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:35.646160 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3f5b6fd8-dc65-46b3-966d-b5db6e58abc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f742cc32a40422ea34d5959731f4444', 'request_id': 'req-3f5b6fd8-dc65-46b3-966d-b5db6e58abc3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:35.696926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3f5b6fd8-dc65-46b3-966d-b5db6e58abc3 admin admin] The token cache is being invalidate because project 4f742cc32a40422ea34d5959731f4444 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:35.702716 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3f5b6fd8-dc65-46b3-966d-b5db6e58abc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:35.705912 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3f5b6fd8-dc65-46b3-966d-b5db6e58abc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:35.714008 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1182/2306] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:35 2022] DELETE /identity/v3/projects/4f742cc32a40422ea34d5959731f4444 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:35.719907 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf6f5c-9582-45af-8c41-35c5ffc1d9f2 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:35.720505 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf6f5c-9582-45af-8c41-35c5ffc1d9f2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:35.720794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf6f5c-9582-45af-8c41-35c5ffc1d9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:35.758104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5cdf6f5c-9582-45af-8c41-35c5ffc1d9f2 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:35.760900 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5cdf6f5c-9582-45af-8c41-35c5ffc1d9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:35.798811 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b6fd8-dc65-46b3-966d-b5db6e58abc3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:35.802802 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3f5b6fd8-dc65-46b3-966d-b5db6e58abc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:35.840015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:35.987685 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1125/2307] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:35.989341 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ad286-671c-445c-996b-3f641dd6c94f None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:35.990423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ad286-671c-445c-996b-3f641dd6c94f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:35.992498 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ad286-671c-445c-996b-3f641dd6c94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:36.026057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5b6fd8-dc65-46b3-966d-b5db6e58abc3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:36.051037 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045617e6-dd7d-4139-b6f0-2b8a6634b0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:36.051843 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045617e6-dd7d-4139-b6f0-2b8a6634b0e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:36.052968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e82ad286-671c-445c-996b-3f641dd6c94f None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:36.054107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045617e6-dd7d-4139-b6f0-2b8a6634b0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:36.054302 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e82ad286-671c-445c-996b-3f641dd6c94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:36.054419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045617e6-dd7d-4139-b6f0-2b8a6634b0e5 admin admin] PATH_INFO: `/v3/projects/973b4a0dd723403b884f5a1050e85a4e` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:36.061796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045617e6-dd7d-4139-b6f0-2b8a6634b0e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=973b4a0dd723403b884f5a1050e85a4e)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:36.063805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045617e6-dd7d-4139-b6f0-2b8a6634b0e5 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:36.085775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1126/2308] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:36.087048 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ad286-671c-445c-996b-3f641dd6c94f None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:36.132136 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ad286-671c-445c-996b-3f641dd6c94f None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:36.174366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-045617e6-dd7d-4139-b6f0-2b8a6634b0e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '973b4a0dd723403b884f5a1050e85a4e', 'request_id': 'req-045617e6-dd7d-4139-b6f0-2b8a6634b0e5'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:36.201576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-045617e6-dd7d-4139-b6f0-2b8a6634b0e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '973b4a0dd723403b884f5a1050e85a4e', 'request_id': 'req-045617e6-dd7d-4139-b6f0-2b8a6634b0e5'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:36.210907 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-045617e6-dd7d-4139-b6f0-2b8a6634b0e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '973b4a0dd723403b884f5a1050e85a4e', 'request_id': 'req-045617e6-dd7d-4139-b6f0-2b8a6634b0e5'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:36.244227 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-045617e6-dd7d-4139-b6f0-2b8a6634b0e5 admin admin] The token cache is being invalidate because project 973b4a0dd723403b884f5a1050e85a4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:36.250952 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-045617e6-dd7d-4139-b6f0-2b8a6634b0e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:36.253781 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-045617e6-dd7d-4139-b6f0-2b8a6634b0e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:36.269436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1183/2309] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:35 2022] DELETE /identity/v3/projects/973b4a0dd723403b884f5a1050e85a4e => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:36.271113 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f702f5e-c3fa-45d9-99c8-415f7b2caf58 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:36.271403 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f702f5e-c3fa-45d9-99c8-415f7b2caf58 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:36.271686 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f702f5e-c3fa-45d9-99c8-415f7b2caf58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:36.307322 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:36.320050 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-3f702f5e-c3fa-45d9-99c8-415f7b2caf58 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:36.321518 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3f702f5e-c3fa-45d9-99c8-415f7b2caf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:36.331360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7cd27-1fae-4884-9f97-72b55e06b46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:36.332705 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7cd27-1fae-4884-9f97-72b55e06b46b admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:36.333204 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7cd27-1fae-4884-9f97-72b55e06b46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:36.333702 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7cd27-1fae-4884-9f97-72b55e06b46b admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:36.334913 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7cd27-1fae-4884-9f97-72b55e06b46b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:34:36.335319 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1184/2310] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:36.335700 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7cd27-1fae-4884-9f97-72b55e06b46b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:36.336277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f702f5e-c3fa-45d9-99c8-415f7b2caf58 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:36.338761 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7cd27-1fae-4884-9f97-72b55e06b46b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:36.357388 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1127/2311] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:34:36.363120 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34c2bb-c9ae-4101-83a8-dc03c5bef119 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:36.363527 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34c2bb-c9ae-4101-83a8-dc03c5bef119 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:36.364488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34c2bb-c9ae-4101-83a8-dc03c5bef119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:36.366194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3f702f5e-c3fa-45d9-99c8-415f7b2caf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:36.396405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f702f5e-c3fa-45d9-99c8-415f7b2caf58 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:36.410993 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599a091-c1af-4538-96fc-55a3ca3f6891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:36.414143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599a091-c1af-4538-96fc-55a3ca3f6891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:36.414505 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599a091-c1af-4538-96fc-55a3ca3f6891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:36.414718 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599a091-c1af-4538-96fc-55a3ca3f6891 admin admin] PATH_INFO: `/v3/users/e7c0f74cbc364b12989f1340fbe6d942` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:36.418811 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a599a091-c1af-4538-96fc-55a3ca3f6891 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7c0f74cbc364b12989f1340fbe6d942)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:36.424614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a599a091-c1af-4538-96fc-55a3ca3f6891 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:36.449002 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-7d34c2bb-c9ae-4101-83a8-dc03c5bef119 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:36.453624 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7d34c2bb-c9ae-4101-83a8-dc03c5bef119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:36.485561 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1128/2312] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:36.487911 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34c2bb-c9ae-4101-83a8-dc03c5bef119 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:36.490139 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7d34c2bb-c9ae-4101-83a8-dc03c5bef119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:36.533889 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34c2bb-c9ae-4101-83a8-dc03c5bef119 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:36.540921 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a599a091-c1af-4538-96fc-55a3ca3f6891 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7c0f74cbc364b12989f1340fbe6d942'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:36.552706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03bbf6-7b8a-40b4-9028-1bf9de5eb6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:36.553330 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03bbf6-7b8a-40b4-9028-1bf9de5eb6de admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:36.553609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03bbf6-7b8a-40b4-9028-1bf9de5eb6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:36.553879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03bbf6-7b8a-40b4-9028-1bf9de5eb6de admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:36.554281 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a599a091-c1af-4538-96fc-55a3ca3f6891 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7c0f74cbc364b12989f1340fbe6d942'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:36.554528 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03bbf6-7b8a-40b4-9028-1bf9de5eb6de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:36.557221 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03bbf6-7b8a-40b4-9028-1bf9de5eb6de admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:36.581028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a599a091-c1af-4538-96fc-55a3ca3f6891 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7c0f74cbc364b12989f1340fbe6d942'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:36.588116 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a599a091-c1af-4538-96fc-55a3ca3f6891 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7c0f74cbc364b12989f1340fbe6d942'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:36.588771 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a599a091-c1af-4538-96fc-55a3ca3f6891 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7c0f74cbc364b12989f1340fbe6d942'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:36.620292 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1129/2313] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:36 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:36.621608 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03bbf6-7b8a-40b4-9028-1bf9de5eb6de admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:36.623519 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-cb03bbf6-7b8a-40b4-9028-1bf9de5eb6de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:36.638346 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1185/2314] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:36 2022] DELETE /identity/v3/users/e7c0f74cbc364b12989f1340fbe6d942 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:36.639492 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599a091-c1af-4538-96fc-55a3ca3f6891 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:36.645051 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a599a091-c1af-4538-96fc-55a3ca3f6891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:36.670085 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:36.752876 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599a091-c1af-4538-96fc-55a3ca3f6891 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:36.765007 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affe001-1c25-49ab-b646-0e5640a201a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:36.766208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affe001-1c25-49ab-b646-0e5640a201a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:36.767605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affe001-1c25-49ab-b646-0e5640a201a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:36.767912 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affe001-1c25-49ab-b646-0e5640a201a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:36.768626 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affe001-1c25-49ab-b646-0e5640a201a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:36.770854 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affe001-1c25-49ab-b646-0e5640a201a9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:36.771383 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03bbf6-7b8a-40b4-9028-1bf9de5eb6de admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:36.796898 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c704d4-9194-4d65-a3a4-2fc5ce855747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:36.796898 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c704d4-9194-4d65-a3a4-2fc5ce855747 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:36.796898 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c704d4-9194-4d65-a3a4-2fc5ce855747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:36.797274 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c704d4-9194-4d65-a3a4-2fc5ce855747 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:36.802396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c704d4-9194-4d65-a3a4-2fc5ce855747 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:34:36.803202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1186/2315] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:36 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:36.804079 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c704d4-9194-4d65-a3a4-2fc5ce855747 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:36.805354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d484ec8-5907-43c3-820a-353f3104c5e3 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:36.806130 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d484ec8-5907-43c3-820a-353f3104c5e3 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:36.806646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d484ec8-5907-43c3-820a-353f3104c5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:36.807794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c704d4-9194-4d65-a3a4-2fc5ce855747 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:36.830020 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2d484ec8-5907-43c3-820a-353f3104c5e3 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:36.831787 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2d484ec8-5907-43c3-820a-353f3104c5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:36.846296 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1187/2316] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:36.847585 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d484ec8-5907-43c3-820a-353f3104c5e3 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:36.852059 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1130/2317] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:34:36.856508 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c704d4-9194-4d65-a3a4-2fc5ce855747 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:36.860874 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d484ec8-5907-43c3-820a-353f3104c5e3 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:36.866882 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f4aad-a482-4892-b771-9e165225d07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:36.868106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f4aad-a482-4892-b771-9e165225d07c admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:36.868539 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f4aad-a482-4892-b771-9e165225d07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:36.869120 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664f4aad-a482-4892-b771-9e165225d07c admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:36.870675 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f4aad-a482-4892-b771-9e165225d07c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:36.872710 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664f4aad-a482-4892-b771-9e165225d07c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:36.878559 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c704d4-9194-4d65-a3a4-2fc5ce855747 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:36.890782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1188/2318] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:36.892673 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f4aad-a482-4892-b771-9e165225d07c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:36.894610 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-664f4aad-a482-4892-b771-9e165225d07c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:36.897251 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d94903-9870-43e5-a27f-f662043b2dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:36.898108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d94903-9870-43e5-a27f-f662043b2dbe admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:36.898399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d94903-9870-43e5-a27f-f662043b2dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:36.898677 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d94903-9870-43e5-a27f-f662043b2dbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:36.900091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d94903-9870-43e5-a27f-f662043b2dbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:36.906570 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d94903-9870-43e5-a27f-f662043b2dbe admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:36.973745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1131/2319] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:36 2022] POST /identity/v3/projects => generated 397 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:36.973745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d94903-9870-43e5-a27f-f662043b2dbe admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:37.062201 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d94903-9870-43e5-a27f-f662043b2dbe admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:37.064793 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664f4aad-a482-4892-b771-9e165225d07c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:37.068699 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188afb2d-7705-49e7-b912-4e364c861cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:37.069391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188afb2d-7705-49e7-b912-4e364c861cca admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:37.069680 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188afb2d-7705-49e7-b912-4e364c861cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:37.070554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188afb2d-7705-49e7-b912-4e364c861cca admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:37.072014 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188afb2d-7705-49e7-b912-4e364c861cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:37.074213 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188afb2d-7705-49e7-b912-4e364c861cca admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:37.089316 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1132/2320] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:37.094896 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188afb2d-7705-49e7-b912-4e364c861cca admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:37.109937 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188afb2d-7705-49e7-b912-4e364c861cca admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:37.121184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad463379-6984-48fe-a837-1d22b009c0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:37.122615 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad463379-6984-48fe-a837-1d22b009c0d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:37.122913 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad463379-6984-48fe-a837-1d22b009c0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:37.123304 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad463379-6984-48fe-a837-1d22b009c0d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:37.124214 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad463379-6984-48fe-a837-1d22b009c0d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:37.127360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad463379-6984-48fe-a837-1d22b009c0d8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:37.169389 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1133/2321] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:37 2022] POST /identity/v3/users => generated 299 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:37.171002 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad463379-6984-48fe-a837-1d22b009c0d8 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:37.200915 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad463379-6984-48fe-a837-1d22b009c0d8 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:37.224782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b4a52-b585-4397-afb4-b42413c769da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:37.225557 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b4a52-b585-4397-afb4-b42413c769da admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:37.225837 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b4a52-b585-4397-afb4-b42413c769da admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:37.226168 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b4a52-b585-4397-afb4-b42413c769da admin admin] PATH_INFO: `/v3/projects/e4de197062eb48adb3f8af1a1f8e958c/users/39abe902a6534e80affc78acafc0ab31/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:37.236222 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:37.265928 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f12fc-6a95-4bfc-b1d7-8292365d02ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:37.266920 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f12fc-6a95-4bfc-b1d7-8292365d02ed service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:37.268705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f12fc-6a95-4bfc-b1d7-8292365d02ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:37.269442 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f12fc-6a95-4bfc-b1d7-8292365d02ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:37.287639 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b4a52-b585-4397-afb4-b42413c769da admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4de197062eb48adb3f8af1a1f8e958c, user_id=39abe902a6534e80affc78acafc0ab31, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:37.291584 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b4a52-b585-4397-afb4-b42413c769da admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:37.343635 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1134/2322] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:37 2022] PUT /identity/v3/projects/e4de197062eb48adb3f8af1a1f8e958c/users/39abe902a6534e80affc78acafc0ab31/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:37.344984 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b4a52-b585-4397-afb4-b42413c769da admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:37.349864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f12fc-6a95-4bfc-b1d7-8292365d02ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:37.352256 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f12fc-6a95-4bfc-b1d7-8292365d02ed service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:37.367575 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1189/2323] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:37.369680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cfb9e-55bf-4e5d-a51e-eed87ffcc9c4 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:37.369975 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cfb9e-55bf-4e5d-a51e-eed87ffcc9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:37.370286 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cfb9e-55bf-4e5d-a51e-eed87ffcc9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:37.477181 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-cf7cfb9e-55bf-4e5d-a51e-eed87ffcc9c4 None None] MFA Rules not processed for user `39abe902a6534e80affc78acafc0ab31`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:37.478814 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-cf7cfb9e-55bf-4e5d-a51e-eed87ffcc9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:37.589067 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b4a52-b585-4397-afb4-b42413c769da admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:37.607532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b5845-1c82-4c6c-9f70-768fb322dc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:37.608441 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b5845-1c82-4c6c-9f70-768fb322dc29 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:37.608948 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b5845-1c82-4c6c-9f70-768fb322dc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:37.609412 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b5845-1c82-4c6c-9f70-768fb322dc29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:37.610341 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b5845-1c82-4c6c-9f70-768fb322dc29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:37.612743 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b5845-1c82-4c6c-9f70-768fb322dc29 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:37.644221 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1135/2324] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:37.650036 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b5845-1c82-4c6c-9f70-768fb322dc29 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:37.670454 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b5845-1c82-4c6c-9f70-768fb322dc29 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:37.712177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:37.725782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb71b8-fa80-4204-ac74-e11710fd8557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:37.726948 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb71b8-fa80-4204-ac74-e11710fd8557 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:37.727382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb71b8-fa80-4204-ac74-e11710fd8557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:37.728338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb71b8-fa80-4204-ac74-e11710fd8557 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:37.735596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb71b8-fa80-4204-ac74-e11710fd8557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:37.750616 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb71b8-fa80-4204-ac74-e11710fd8557 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:37.772788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1136/2325] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:37.777654 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb71b8-fa80-4204-ac74-e11710fd8557 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:37.796386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb71b8-fa80-4204-ac74-e11710fd8557 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:37.806398 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42d9fd-dae0-4200-85ff-2b72af04b7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:37.808067 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42d9fd-dae0-4200-85ff-2b72af04b7c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:37.808670 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42d9fd-dae0-4200-85ff-2b72af04b7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:37.809210 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42d9fd-dae0-4200-85ff-2b72af04b7c6 admin admin] PATH_INFO: `/v3/projects/88ef468ef8294bcab8d4c98a9ab3bd86/users/75373b0da8bf4d4dad2a9e809e7d0867/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:37.838463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1190/2326] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:37.841194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cfb9e-55bf-4e5d-a51e-eed87ffcc9c4 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:37.843067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-cf7cfb9e-55bf-4e5d-a51e-eed87ffcc9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:37.858699 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42d9fd-dae0-4200-85ff-2b72af04b7c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88ef468ef8294bcab8d4c98a9ab3bd86, user_id=75373b0da8bf4d4dad2a9e809e7d0867, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:37.862070 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42d9fd-dae0-4200-85ff-2b72af04b7c6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:37.915109 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1137/2327] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:37 2022] PUT /identity/v3/projects/88ef468ef8294bcab8d4c98a9ab3bd86/users/75373b0da8bf4d4dad2a9e809e7d0867/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:37.934902 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdd3c1-7508-41b4-85ae-813fbb258681 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:37.936626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdd3c1-7508-41b4-85ae-813fbb258681 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:37.937060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdd3c1-7508-41b4-85ae-813fbb258681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:38.042964 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-a5fdd3c1-7508-41b4-85ae-813fbb258681 None None] MFA Rules not processed for user `75373b0da8bf4d4dad2a9e809e7d0867`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:38.044660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a5fdd3c1-7508-41b4-85ae-813fbb258681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:38.054616 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cfb9e-55bf-4e5d-a51e-eed87ffcc9c4 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:38.188976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b63a3-c827-410c-a918-198e8a2734bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:38.190107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b63a3-c827-410c-a918-198e8a2734bd service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:38.190941 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b63a3-c827-410c-a918-198e8a2734bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:38.191222 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b63a3-c827-410c-a918-198e8a2734bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:38.253540 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1138/2328] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:37 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:38.271968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fdd3c1-7508-41b4-85ae-813fbb258681 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:38.273681 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a5fdd3c1-7508-41b4-85ae-813fbb258681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:38.324284 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fdd3c1-7508-41b4-85ae-813fbb258681 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:38.344055 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b1148-cf2a-4ed5-9648-00dac7fc6d0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:38.344880 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b1148-cf2a-4ed5-9648-00dac7fc6d0a service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:38.345255 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b1148-cf2a-4ed5-9648-00dac7fc6d0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:38.346148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b1148-cf2a-4ed5-9648-00dac7fc6d0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:38.381347 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b1148-cf2a-4ed5-9648-00dac7fc6d0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:38.383560 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b1148-cf2a-4ed5-9648-00dac7fc6d0a service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:38.383778 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b63a3-c827-410c-a918-198e8a2734bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:38.385271 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b63a3-c827-410c-a918-198e8a2734bd service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:38.414883 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1191/2329] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:38.425681 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1139/2330] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:38 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:34:38.794021 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385502a9-150b-48b2-9780-3967c7e6de3d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:38.794669 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385502a9-150b-48b2-9780-3967c7e6de3d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:38.794948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385502a9-150b-48b2-9780-3967c7e6de3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:38.884855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b1148-cf2a-4ed5-9648-00dac7fc6d0a service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:38.889856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e49b1148-cf2a-4ed5-9648-00dac7fc6d0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:38.997562 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-385502a9-150b-48b2-9780-3967c7e6de3d None None] MFA Rules not processed for user `448dc80e8c1b486fbdf83317209528eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:39.000223 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-385502a9-150b-48b2-9780-3967c7e6de3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:39.061897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:39.150237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:39.263292 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b1148-cf2a-4ed5-9648-00dac7fc6d0a service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:39.282736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ed895-e7b5-4277-97b2-115b22b4ffbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:39.283717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ed895-e7b5-4277-97b2-115b22b4ffbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:39.283717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ed895-e7b5-4277-97b2-115b22b4ffbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:39.291253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ed895-e7b5-4277-97b2-115b22b4ffbd admin admin] PATH_INFO: `/v3/projects/7b1586a95edd4b04943d104f9f814757` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:39.299291 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ed895-e7b5-4277-97b2-115b22b4ffbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b1586a95edd4b04943d104f9f814757)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:39.312068 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ed895-e7b5-4277-97b2-115b22b4ffbd admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:39.388339 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1192/2331] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 596 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:39.394841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385502a9-150b-48b2-9780-3967c7e6de3d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:39.398276 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-385502a9-150b-48b2-9780-3967c7e6de3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:39.452582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4a7ed895-e7b5-4277-97b2-115b22b4ffbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b1586a95edd4b04943d104f9f814757', 'request_id': 'req-4a7ed895-e7b5-4277-97b2-115b22b4ffbd'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:39.468211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385502a9-150b-48b2-9780-3967c7e6de3d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:39.483218 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4a7ed895-e7b5-4277-97b2-115b22b4ffbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b1586a95edd4b04943d104f9f814757', 'request_id': 'req-4a7ed895-e7b5-4277-97b2-115b22b4ffbd'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:39.516167 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4a7ed895-e7b5-4277-97b2-115b22b4ffbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b1586a95edd4b04943d104f9f814757', 'request_id': 'req-4a7ed895-e7b5-4277-97b2-115b22b4ffbd'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:39.591819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4a7ed895-e7b5-4277-97b2-115b22b4ffbd admin admin] The token cache is being invalidate because project 7b1586a95edd4b04943d104f9f814757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:39.593025 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4a7ed895-e7b5-4277-97b2-115b22b4ffbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:39.594095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4a7ed895-e7b5-4277-97b2-115b22b4ffbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:39.599567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1140/2332] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:38 2022] DELETE /identity/v3/projects/7b1586a95edd4b04943d104f9f814757 => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:39.603505 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ed895-e7b5-4277-97b2-115b22b4ffbd admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:39.605121 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63423474-1de3-4eaa-862b-1bebadd34f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:39.606175 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63423474-1de3-4eaa-862b-1bebadd34f6b admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:39.606454 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4a7ed895-e7b5-4277-97b2-115b22b4ffbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:39.606889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63423474-1de3-4eaa-862b-1bebadd34f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:39.608044 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63423474-1de3-4eaa-862b-1bebadd34f6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:39.609233 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63423474-1de3-4eaa-862b-1bebadd34f6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:39.612031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63423474-1de3-4eaa-862b-1bebadd34f6b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:39.630217 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ed895-e7b5-4277-97b2-115b22b4ffbd admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:39.637440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43df81-1181-4d11-ba84-d604959167c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:39.638390 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43df81-1181-4d11-ba84-d604959167c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:39.638574 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43df81-1181-4d11-ba84-d604959167c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:39.638954 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43df81-1181-4d11-ba84-d604959167c5 admin admin] PATH_INFO: `/v3/users/a8036dbc918747b2898113836a301421` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:39.640495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c43df81-1181-4d11-ba84-d604959167c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8036dbc918747b2898113836a301421)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:39.642803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c43df81-1181-4d11-ba84-d604959167c5 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:39.664622 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1193/2333] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:39 2022] POST /identity/v3/projects => generated 383 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:39.666401 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63423474-1de3-4eaa-862b-1bebadd34f6b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:39.669189 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-63423474-1de3-4eaa-862b-1bebadd34f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:39.864567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8c43df81-1181-4d11-ba84-d604959167c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8036dbc918747b2898113836a301421'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:39.887848 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8c43df81-1181-4d11-ba84-d604959167c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8036dbc918747b2898113836a301421'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:39.901791 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8c43df81-1181-4d11-ba84-d604959167c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8036dbc918747b2898113836a301421'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:39.902751 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8c43df81-1181-4d11-ba84-d604959167c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8036dbc918747b2898113836a301421'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:39.913756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63423474-1de3-4eaa-862b-1bebadd34f6b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:39.935453 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01033beb-1f57-45ef-9b50-5903e911ef88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:39.936236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01033beb-1f57-45ef-9b50-5903e911ef88 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:39.936556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01033beb-1f57-45ef-9b50-5903e911ef88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:39.937136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01033beb-1f57-45ef-9b50-5903e911ef88 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:39.938271 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01033beb-1f57-45ef-9b50-5903e911ef88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:39.940287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01033beb-1f57-45ef-9b50-5903e911ef88 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:39.941128 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-8c43df81-1181-4d11-ba84-d604959167c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8036dbc918747b2898113836a301421'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:39.955314 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1141/2334] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:39 2022] DELETE /identity/v3/users/a8036dbc918747b2898113836a301421 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:40.005900 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1194/2335] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:39 2022] POST /identity/v3/users => generated 298 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:40.010847 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43df81-1181-4d11-ba84-d604959167c5 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:40.012542 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8c43df81-1181-4d11-ba84-d604959167c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:40.048105 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:40.134714 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43df81-1181-4d11-ba84-d604959167c5 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:40.152807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57719d8e-cd8b-4bc0-a81f-c6cab3b6a045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:40.153437 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57719d8e-cd8b-4bc0-a81f-c6cab3b6a045 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:40.153713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57719d8e-cd8b-4bc0-a81f-c6cab3b6a045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:40.153983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57719d8e-cd8b-4bc0-a81f-c6cab3b6a045 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:40.154630 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57719d8e-cd8b-4bc0-a81f-c6cab3b6a045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:40.159913 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57719d8e-cd8b-4bc0-a81f-c6cab3b6a045 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:40.200501 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1142/2336] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:40.202712 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01033beb-1f57-45ef-9b50-5903e911ef88 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:40.220901 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01033beb-1f57-45ef-9b50-5903e911ef88 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:40.236495 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952d644-8c20-483c-92b1-1463533d860b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:40.237269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952d644-8c20-483c-92b1-1463533d860b admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:40.237673 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952d644-8c20-483c-92b1-1463533d860b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:40.237813 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952d644-8c20-483c-92b1-1463533d860b admin admin] PATH_INFO: `/v3/projects/fd7f1416e786402898e6377e1c3f210c/users/4c5ce3754d6d4af8a325683564c48538/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:40.272974 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952d644-8c20-483c-92b1-1463533d860b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7f1416e786402898e6377e1c3f210c, user_id=4c5ce3754d6d4af8a325683564c48538, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:40.278515 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952d644-8c20-483c-92b1-1463533d860b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:40.315457 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1195/2337] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:40 2022] PUT /identity/v3/projects/fd7f1416e786402898e6377e1c3f210c/users/4c5ce3754d6d4af8a325683564c48538/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:40.317317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57719d8e-cd8b-4bc0-a81f-c6cab3b6a045 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:40.331703 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57719d8e-cd8b-4bc0-a81f-c6cab3b6a045 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:40.523886 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291ee8d-f09d-40be-82cb-2cc9d1859e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:40.524658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291ee8d-f09d-40be-82cb-2cc9d1859e65 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:40.524945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291ee8d-f09d-40be-82cb-2cc9d1859e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:40.525221 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291ee8d-f09d-40be-82cb-2cc9d1859e65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:40.525891 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1291ee8d-f09d-40be-82cb-2cc9d1859e65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:40.527542 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1291ee8d-f09d-40be-82cb-2cc9d1859e65 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:40.557758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1143/2338] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:40 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:34:40.559105 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952d644-8c20-483c-92b1-1463533d860b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:40.577768 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952d644-8c20-483c-92b1-1463533d860b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:40.591727 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985040e-4da0-4637-aae2-484e93334e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:40.594021 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985040e-4da0-4637-aae2-484e93334e55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:40.594021 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985040e-4da0-4637-aae2-484e93334e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:40.594021 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5985040e-4da0-4637-aae2-484e93334e55 admin admin] PATH_INFO: `/v3/projects/fd7f1416e786402898e6377e1c3f210c/users/4c5ce3754d6d4af8a325683564c48538/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:40.606162 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5985040e-4da0-4637-aae2-484e93334e55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7f1416e786402898e6377e1c3f210c, user_id=4c5ce3754d6d4af8a325683564c48538, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:40.614456 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5985040e-4da0-4637-aae2-484e93334e55 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:40.661265 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b18e7-2d8d-4f43-997f-9671a5fae26c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:40.661581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b18e7-2d8d-4f43-997f-9671a5fae26c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:40.661858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b18e7-2d8d-4f43-997f-9671a5fae26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:40.672972 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1196/2339] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:40 2022] PUT /identity/v3/projects/fd7f1416e786402898e6377e1c3f210c/users/4c5ce3754d6d4af8a325683564c48538/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:40.713589 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ca2b18e7-2d8d-4f43-997f-9671a5fae26c None None] MFA Rules not processed for user `39abe902a6534e80affc78acafc0ab31`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:40.715383 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ca2b18e7-2d8d-4f43-997f-9671a5fae26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:40.929570 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1144/2340] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:34:41.149388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985040e-4da0-4637-aae2-484e93334e55 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:41.152921 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5985040e-4da0-4637-aae2-484e93334e55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:41.199103 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:41.498673 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5985040e-4da0-4637-aae2-484e93334e55 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:41.510686 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d312ee8-582f-44fd-b80e-29b49dbfe0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:41.517214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d312ee8-582f-44fd-b80e-29b49dbfe0a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:41.517761 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d312ee8-582f-44fd-b80e-29b49dbfe0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:41.518053 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d312ee8-582f-44fd-b80e-29b49dbfe0a6 admin admin] PATH_INFO: `/v3/projects/7448a1301ade4188811365e4b7e580a0` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:41.519917 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d312ee8-582f-44fd-b80e-29b49dbfe0a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7448a1301ade4188811365e4b7e580a0)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:41.522387 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d312ee8-582f-44fd-b80e-29b49dbfe0a6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:41.667674 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8d312ee8-582f-44fd-b80e-29b49dbfe0a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7448a1301ade4188811365e4b7e580a0', 'request_id': 'req-8d312ee8-582f-44fd-b80e-29b49dbfe0a6'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:41.690678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8d312ee8-582f-44fd-b80e-29b49dbfe0a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7448a1301ade4188811365e4b7e580a0', 'request_id': 'req-8d312ee8-582f-44fd-b80e-29b49dbfe0a6'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:41.697513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8d312ee8-582f-44fd-b80e-29b49dbfe0a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7448a1301ade4188811365e4b7e580a0', 'request_id': 'req-8d312ee8-582f-44fd-b80e-29b49dbfe0a6'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:41.722655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8d312ee8-582f-44fd-b80e-29b49dbfe0a6 admin admin] The token cache is being invalidate because project 7448a1301ade4188811365e4b7e580a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:41.723008 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8d312ee8-582f-44fd-b80e-29b49dbfe0a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:41.724837 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-8d312ee8-582f-44fd-b80e-29b49dbfe0a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:41.733754 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1197/2341] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:41 2022] DELETE /identity/v3/projects/7448a1301ade4188811365e4b7e580a0 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:41.743209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249c382-bd19-4a92-9164-d14ead3c5bea None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:41.744371 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249c382-bd19-4a92-9164-d14ead3c5bea None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:41.745102 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249c382-bd19-4a92-9164-d14ead3c5bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:41.826371 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c249c382-bd19-4a92-9164-d14ead3c5bea None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:41.830818 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c249c382-bd19-4a92-9164-d14ead3c5bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:42.092460 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:42.127058 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1145/2342] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:42.127990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f2dc0-9ec3-4a14-818a-b8c49935d53a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:42.128279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f2dc0-9ec3-4a14-818a-b8c49935d53a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:42.128617 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f2dc0-9ec3-4a14-818a-b8c49935d53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:42.170643 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-471f2dc0-9ec3-4a14-818a-b8c49935d53a None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:42.171877 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-471f2dc0-9ec3-4a14-818a-b8c49935d53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:42.194325 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1198/2343] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:34:42.201445 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249c382-bd19-4a92-9164-d14ead3c5bea None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:42.252137 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249c382-bd19-4a92-9164-d14ead3c5bea None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:42.291796 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff305d76-5c8d-4562-8fa5-20e987a41046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:42.295022 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff305d76-5c8d-4562-8fa5-20e987a41046 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:42.296076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff305d76-5c8d-4562-8fa5-20e987a41046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:42.296794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff305d76-5c8d-4562-8fa5-20e987a41046 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:42.298827 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff305d76-5c8d-4562-8fa5-20e987a41046 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:34:42.305627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff305d76-5c8d-4562-8fa5-20e987a41046 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:42.315983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff305d76-5c8d-4562-8fa5-20e987a41046 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:42.370474 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f2dc0-9ec3-4a14-818a-b8c49935d53a None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:42.387859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1146/2344] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:34:42.394622 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f2dc0-9ec3-4a14-818a-b8c49935d53a None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:42.421459 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9b1c9-7ff7-4ee2-a57a-0954bcdf97fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:42.422485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9b1c9-7ff7-4ee2-a57a-0954bcdf97fd admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:42.423140 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9b1c9-7ff7-4ee2-a57a-0954bcdf97fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:42.423486 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9b1c9-7ff7-4ee2-a57a-0954bcdf97fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:42.424176 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9b1c9-7ff7-4ee2-a57a-0954bcdf97fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:42.428796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9b1c9-7ff7-4ee2-a57a-0954bcdf97fd admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:42.548043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1199/2345] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:42 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:42.548846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff305d76-5c8d-4562-8fa5-20e987a41046 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:42.564523 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff305d76-5c8d-4562-8fa5-20e987a41046 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:42.687836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b040a8-fad8-46e2-91b6-a78f61c3823c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:42.688952 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b040a8-fad8-46e2-91b6-a78f61c3823c admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:42.689247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b040a8-fad8-46e2-91b6-a78f61c3823c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:42.689665 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b040a8-fad8-46e2-91b6-a78f61c3823c admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:42.690266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b040a8-fad8-46e2-91b6-a78f61c3823c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:42.692509 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b040a8-fad8-46e2-91b6-a78f61c3823c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:42.724329 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1147/2346] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:42 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:42.730782 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9b1c9-7ff7-4ee2-a57a-0954bcdf97fd admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:42.747286 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9b1c9-7ff7-4ee2-a57a-0954bcdf97fd admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:42.754189 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b135b83-b538-4d8d-a24f-78baa51e630a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:42.755477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b135b83-b538-4d8d-a24f-78baa51e630a admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:42.755762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b135b83-b538-4d8d-a24f-78baa51e630a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:42.756121 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b135b83-b538-4d8d-a24f-78baa51e630a admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:42.757645 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b135b83-b538-4d8d-a24f-78baa51e630a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:42.760762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b135b83-b538-4d8d-a24f-78baa51e630a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:42.791321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1200/2347] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:34:42.792880 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b040a8-fad8-46e2-91b6-a78f61c3823c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:42.811735 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b040a8-fad8-46e2-91b6-a78f61c3823c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:42.847564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b278f0-5404-4a35-b911-8a25a15da5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:42.848687 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b278f0-5404-4a35-b911-8a25a15da5d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:42.849261 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b278f0-5404-4a35-b911-8a25a15da5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:42.849759 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b278f0-5404-4a35-b911-8a25a15da5d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:42.851370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b278f0-5404-4a35-b911-8a25a15da5d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:42.863466 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b278f0-5404-4a35-b911-8a25a15da5d3 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:42.895148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1148/2348] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:42.896503 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b135b83-b538-4d8d-a24f-78baa51e630a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:42.912363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b135b83-b538-4d8d-a24f-78baa51e630a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:42.931108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07910991-4fe5-4221-85ba-ec763d06d9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:42.933201 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07910991-4fe5-4221-85ba-ec763d06d9ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:42.933542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07910991-4fe5-4221-85ba-ec763d06d9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:42.936074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07910991-4fe5-4221-85ba-ec763d06d9ba admin admin] PATH_INFO: `/v3/projects/e864364ee58f4caa8c06c04da26887d1/users/74025a93b23642939b155fe2c54ddd03/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:42.995060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07910991-4fe5-4221-85ba-ec763d06d9ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e864364ee58f4caa8c06c04da26887d1, user_id=74025a93b23642939b155fe2c54ddd03, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:43.000719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07910991-4fe5-4221-85ba-ec763d06d9ba admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:43.038314 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1201/2349] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:42 2022] PUT /identity/v3/projects/e864364ee58f4caa8c06c04da26887d1/users/74025a93b23642939b155fe2c54ddd03/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:43.052701 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976f515-b522-48eb-a23e-a194c38d0f15 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:43.053534 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976f515-b522-48eb-a23e-a194c38d0f15 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:43.054047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976f515-b522-48eb-a23e-a194c38d0f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:43.161840 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d976f515-b522-48eb-a23e-a194c38d0f15 None None] MFA Rules not processed for user `74025a93b23642939b155fe2c54ddd03`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:43.169181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d976f515-b522-48eb-a23e-a194c38d0f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:43.339749 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1149/2350] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:34:43.364035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07910991-4fe5-4221-85ba-ec763d06d9ba admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:43.368951 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-07910991-4fe5-4221-85ba-ec763d06d9ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:43.466095 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:43.507835 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07910991-4fe5-4221-85ba-ec763d06d9ba admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:43.515203 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fb24b-1e74-482a-bc24-aaba9ec42303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:43.516116 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fb24b-1e74-482a-bc24-aaba9ec42303 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:43.518064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fb24b-1e74-482a-bc24-aaba9ec42303 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:43.518527 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fb24b-1e74-482a-bc24-aaba9ec42303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:43.531234 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2fb24b-1e74-482a-bc24-aaba9ec42303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:43.533108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2fb24b-1e74-482a-bc24-aaba9ec42303 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:43.549972 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1202/2351] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:34:43.629953 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976f515-b522-48eb-a23e-a194c38d0f15 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:43.822429 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:43.831182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976f515-b522-48eb-a23e-a194c38d0f15 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:43.839433 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a5857-c403-43ea-89ff-e8fc73506446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:43.840316 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a5857-c403-43ea-89ff-e8fc73506446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:43.840653 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a5857-c403-43ea-89ff-e8fc73506446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:43.843272 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a5857-c403-43ea-89ff-e8fc73506446 admin admin] PATH_INFO: `/v3/users/74025a93b23642939b155fe2c54ddd03` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:43.845591 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a5857-c403-43ea-89ff-e8fc73506446 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74025a93b23642939b155fe2c54ddd03)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:43.847687 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a5857-c403-43ea-89ff-e8fc73506446 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:44.004746 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-502a5857-c403-43ea-89ff-e8fc73506446 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74025a93b23642939b155fe2c54ddd03'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:44.034386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-502a5857-c403-43ea-89ff-e8fc73506446 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74025a93b23642939b155fe2c54ddd03'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:44.054641 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-502a5857-c403-43ea-89ff-e8fc73506446 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74025a93b23642939b155fe2c54ddd03'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:44.056748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-502a5857-c403-43ea-89ff-e8fc73506446 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74025a93b23642939b155fe2c54ddd03'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:44.089000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-502a5857-c403-43ea-89ff-e8fc73506446 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74025a93b23642939b155fe2c54ddd03'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:44.110581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1150/2352] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:43 2022] DELETE /identity/v3/users/74025a93b23642939b155fe2c54ddd03 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:44.114576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767abb9-10cd-4e17-aea3-9ea7cf39ac24 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:44.114876 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767abb9-10cd-4e17-aea3-9ea7cf39ac24 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:44.115164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767abb9-10cd-4e17-aea3-9ea7cf39ac24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:44.179122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b767abb9-10cd-4e17-aea3-9ea7cf39ac24 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:44.184615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b767abb9-10cd-4e17-aea3-9ea7cf39ac24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:44.401274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1203/2353] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:44.445262 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a5857-c403-43ea-89ff-e8fc73506446 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:44.451713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-502a5857-c403-43ea-89ff-e8fc73506446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:44.640509 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a5857-c403-43ea-89ff-e8fc73506446 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:44.652983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3dadd0-9234-4afe-a022-8e8f6c8ae2bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:44.653633 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3dadd0-9234-4afe-a022-8e8f6c8ae2bc service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:44.653923 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3dadd0-9234-4afe-a022-8e8f6c8ae2bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:44.654187 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3dadd0-9234-4afe-a022-8e8f6c8ae2bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:44.668811 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3dadd0-9234-4afe-a022-8e8f6c8ae2bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:44.670293 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3dadd0-9234-4afe-a022-8e8f6c8ae2bc service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:44.690252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1151/2354] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:45.239071 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481dbe8-ac36-4dcd-95e9-37f95afae8f1 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:45.239320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481dbe8-ac36-4dcd-95e9-37f95afae8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:45.239545 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481dbe8-ac36-4dcd-95e9-37f95afae8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:45.328535 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d481dbe8-ac36-4dcd-95e9-37f95afae8f1 None None] MFA Rules not processed for user `068f28eb74e74650a2b80d29664109ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:45.337722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d481dbe8-ac36-4dcd-95e9-37f95afae8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:45.491228 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1204/2355] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:45.511908 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3dadd0-9234-4afe-a022-8e8f6c8ae2bc service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:45.513377 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ca3dadd0-9234-4afe-a022-8e8f6c8ae2bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:45.592973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481dbe8-ac36-4dcd-95e9-37f95afae8f1 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:45.592973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d481dbe8-ac36-4dcd-95e9-37f95afae8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:45.651716 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481dbe8-ac36-4dcd-95e9-37f95afae8f1 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:45.665203 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f1658-3a57-409b-a0f5-145d6b3b9fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:45.665979 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f1658-3a57-409b-a0f5-145d6b3b9fdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:45.666345 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f1658-3a57-409b-a0f5-145d6b3b9fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:45.666628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f1658-3a57-409b-a0f5-145d6b3b9fdc admin admin] PATH_INFO: `/v3/projects/e864364ee58f4caa8c06c04da26887d1` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:45.671601 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f1658-3a57-409b-a0f5-145d6b3b9fdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e864364ee58f4caa8c06c04da26887d1)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:45.678342 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3dadd0-9234-4afe-a022-8e8f6c8ae2bc service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:45.682110 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f1658-3a57-409b-a0f5-145d6b3b9fdc admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:45.695554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5bf05-95ca-4b17-b8f4-27aacd780359 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:45.696210 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5bf05-95ca-4b17-b8f4-27aacd780359 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:45.696501 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5bf05-95ca-4b17-b8f4-27aacd780359 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:45.697382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5bf05-95ca-4b17-b8f4-27aacd780359 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:45.714856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5bf05-95ca-4b17-b8f4-27aacd780359 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:45.716584 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5bf05-95ca-4b17-b8f4-27aacd780359 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:45.738191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1152/2356] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:45.932066 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-349f1658-3a57-409b-a0f5-145d6b3b9fdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e864364ee58f4caa8c06c04da26887d1', 'request_id': 'req-349f1658-3a57-409b-a0f5-145d6b3b9fdc'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:45.964933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-349f1658-3a57-409b-a0f5-145d6b3b9fdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e864364ee58f4caa8c06c04da26887d1', 'request_id': 'req-349f1658-3a57-409b-a0f5-145d6b3b9fdc'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:45.974633 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:45.983368 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-349f1658-3a57-409b-a0f5-145d6b3b9fdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e864364ee58f4caa8c06c04da26887d1', 'request_id': 'req-349f1658-3a57-409b-a0f5-145d6b3b9fdc'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:46.022952 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-349f1658-3a57-409b-a0f5-145d6b3b9fdc admin admin] The token cache is being invalidate because project e864364ee58f4caa8c06c04da26887d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:46.023315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-349f1658-3a57-409b-a0f5-145d6b3b9fdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:46.025114 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-349f1658-3a57-409b-a0f5-145d6b3b9fdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:46.030357 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1205/2357] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:45 2022] DELETE /identity/v3/projects/e864364ee58f4caa8c06c04da26887d1 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:46.032565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d718edc-b9c9-4e03-8c3d-6e074b0f3add None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:46.033247 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d718edc-b9c9-4e03-8c3d-6e074b0f3add None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:46.033514 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d718edc-b9c9-4e03-8c3d-6e074b0f3add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:46.059526 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:46.089629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-8d718edc-b9c9-4e03-8c3d-6e074b0f3add None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:46.090910 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8d718edc-b9c9-4e03-8c3d-6e074b0f3add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:46.226121 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1153/2358] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:46.233157 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fde90-af6a-490c-8770-f6ef3749394a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:46.235046 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fde90-af6a-490c-8770-f6ef3749394a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:46.235753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fde90-af6a-490c-8770-f6ef3749394a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:46.316142 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ec0fde90-af6a-490c-8770-f6ef3749394a None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:46.319695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ec0fde90-af6a-490c-8770-f6ef3749394a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:46.390005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1206/2359] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:34:46.395739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d718edc-b9c9-4e03-8c3d-6e074b0f3add None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:46.412437 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d718edc-b9c9-4e03-8c3d-6e074b0f3add None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:46.425906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576b83a-0076-479c-b06f-0159c980a30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:46.426210 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576b83a-0076-479c-b06f-0159c980a30e admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:46.426585 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576b83a-0076-479c-b06f-0159c980a30e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:46.426851 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576b83a-0076-479c-b06f-0159c980a30e admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:46.427557 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576b83a-0076-479c-b06f-0159c980a30e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:34:46.427888 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576b83a-0076-479c-b06f-0159c980a30e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:46.429662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576b83a-0076-479c-b06f-0159c980a30e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:46.468373 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1154/2360] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:34:46.469126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fde90-af6a-490c-8770-f6ef3749394a None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:46.485381 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fde90-af6a-490c-8770-f6ef3749394a None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:46.508361 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb04abf-ee24-4c8d-91b3-d7b4b1e11230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:46.510226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb04abf-ee24-4c8d-91b3-d7b4b1e11230 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:46.510226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb04abf-ee24-4c8d-91b3-d7b4b1e11230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:46.527627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb04abf-ee24-4c8d-91b3-d7b4b1e11230 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:46.529736 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb04abf-ee24-4c8d-91b3-d7b4b1e11230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:46.554844 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb04abf-ee24-4c8d-91b3-d7b4b1e11230 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:46.694567 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1207/2361] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:46 2022] POST /identity/v3/projects => generated 375 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:34:46.696925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576b83a-0076-479c-b06f-0159c980a30e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:46.716938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576b83a-0076-479c-b06f-0159c980a30e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:46.927985 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d7fba-a0f0-4b93-87e5-027dd280e09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:46.929095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d7fba-a0f0-4b93-87e5-027dd280e09a admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:46.929606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d7fba-a0f0-4b93-87e5-027dd280e09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:46.930116 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d7fba-a0f0-4b93-87e5-027dd280e09a admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:46.931422 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d7fba-a0f0-4b93-87e5-027dd280e09a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:46.933799 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d7fba-a0f0-4b93-87e5-027dd280e09a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:46.986638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1155/2362] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:46 2022] POST /identity/v3/users => generated 288 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:46.991351 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb04abf-ee24-4c8d-91b3-d7b4b1e11230 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:47.004944 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb04abf-ee24-4c8d-91b3-d7b4b1e11230 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:47.015197 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd2af7-81dc-4dbe-8105-f0a11cee95fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:47.015973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd2af7-81dc-4dbe-8105-f0a11cee95fd admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:47.018298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd2af7-81dc-4dbe-8105-f0a11cee95fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:47.018478 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd2af7-81dc-4dbe-8105-f0a11cee95fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:47.019270 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd2af7-81dc-4dbe-8105-f0a11cee95fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:47.021605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd2af7-81dc-4dbe-8105-f0a11cee95fd admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:47.093560 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1208/2363] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:47.104846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d7fba-a0f0-4b93-87e5-027dd280e09a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:47.120387 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d7fba-a0f0-4b93-87e5-027dd280e09a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:47.126945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29a1c1-5b51-4ce6-af96-0154bce46164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:47.127679 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29a1c1-5b51-4ce6-af96-0154bce46164 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:47.128131 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29a1c1-5b51-4ce6-af96-0154bce46164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:47.128502 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29a1c1-5b51-4ce6-af96-0154bce46164 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:47.129416 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29a1c1-5b51-4ce6-af96-0154bce46164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:47.132115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29a1c1-5b51-4ce6-af96-0154bce46164 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:47.164232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1156/2364] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:47.170391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd2af7-81dc-4dbe-8105-f0a11cee95fd admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:47.196154 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd2af7-81dc-4dbe-8105-f0a11cee95fd admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:47.210655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b523ef-6b86-434e-8314-50045f8253e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:47.211442 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b523ef-6b86-434e-8314-50045f8253e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:47.211790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b523ef-6b86-434e-8314-50045f8253e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:47.212747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b523ef-6b86-434e-8314-50045f8253e1 admin admin] PATH_INFO: `/v3/projects/7f91610898f74be3b832cc55807e4a7d/users/34e80e67c7d34441966451c0de3c2a2e/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:47.244344 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b523ef-6b86-434e-8314-50045f8253e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f91610898f74be3b832cc55807e4a7d, user_id=34e80e67c7d34441966451c0de3c2a2e, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:47.247826 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b523ef-6b86-434e-8314-50045f8253e1 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:47.268982 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1209/2365] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:47 2022] PUT /identity/v3/projects/7f91610898f74be3b832cc55807e4a7d/users/34e80e67c7d34441966451c0de3c2a2e/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:47.294472 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aac14c-e5e2-41d1-a18e-9f515b925a77 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:47.295527 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aac14c-e5e2-41d1-a18e-9f515b925a77 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:47.296039 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aac14c-e5e2-41d1-a18e-9f515b925a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:47.357900 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-97aac14c-e5e2-41d1-a18e-9f515b925a77 None None] MFA Rules not processed for user `34e80e67c7d34441966451c0de3c2a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:47.359074 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-97aac14c-e5e2-41d1-a18e-9f515b925a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:47.472248 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:47.492583 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1157/2366] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:47.504268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b523ef-6b86-434e-8314-50045f8253e1 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:47.506817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-00b523ef-6b86-434e-8314-50045f8253e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:47.687305 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b523ef-6b86-434e-8314-50045f8253e1 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:47.703359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68963fd1-d4a6-4a10-925a-5d8c0f1c8fce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:47.704802 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68963fd1-d4a6-4a10-925a-5d8c0f1c8fce service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:47.705587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68963fd1-d4a6-4a10-925a-5d8c0f1c8fce service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:47.706259 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68963fd1-d4a6-4a10-925a-5d8c0f1c8fce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:47.745313 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68963fd1-d4a6-4a10-925a-5d8c0f1c8fce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:47.746830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68963fd1-d4a6-4a10-925a-5d8c0f1c8fce service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:47.770900 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1210/2367] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:48.506879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aac14c-e5e2-41d1-a18e-9f515b925a77 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:48.509056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-97aac14c-e5e2-41d1-a18e-9f515b925a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:48.700456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aac14c-e5e2-41d1-a18e-9f515b925a77 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:48.719130 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731c35f-a02e-47f0-bf64-b36d367351ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:48.723486 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731c35f-a02e-47f0-bf64-b36d367351ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:48.723847 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731c35f-a02e-47f0-bf64-b36d367351ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:48.724177 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731c35f-a02e-47f0-bf64-b36d367351ea admin admin] PATH_INFO: `/v3/users/75373b0da8bf4d4dad2a9e809e7d0867` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:48.728271 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731c35f-a02e-47f0-bf64-b36d367351ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=75373b0da8bf4d4dad2a9e809e7d0867)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:48.732754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731c35f-a02e-47f0-bf64-b36d367351ea admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:48.881280 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0731c35f-a02e-47f0-bf64-b36d367351ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75373b0da8bf4d4dad2a9e809e7d0867'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:48.899497 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0731c35f-a02e-47f0-bf64-b36d367351ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75373b0da8bf4d4dad2a9e809e7d0867'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:48.917696 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0731c35f-a02e-47f0-bf64-b36d367351ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75373b0da8bf4d4dad2a9e809e7d0867'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:48.920725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0731c35f-a02e-47f0-bf64-b36d367351ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75373b0da8bf4d4dad2a9e809e7d0867'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:48.956702 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0731c35f-a02e-47f0-bf64-b36d367351ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75373b0da8bf4d4dad2a9e809e7d0867'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:48.996337 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1158/2368] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:48 2022] DELETE /identity/v3/users/75373b0da8bf4d4dad2a9e809e7d0867 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:48.998072 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e54b2-be93-47ad-88d6-951baaa8d031 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:48.998777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e54b2-be93-47ad-88d6-951baaa8d031 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:48.999113 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e54b2-be93-47ad-88d6-951baaa8d031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:49.043489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d29e54b2-be93-47ad-88d6-951baaa8d031 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:49.044851 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d29e54b2-be93-47ad-88d6-951baaa8d031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:49.142535 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1211/2369] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:49.316191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731c35f-a02e-47f0-bf64-b36d367351ea admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:49.317931 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0731c35f-a02e-47f0-bf64-b36d367351ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:49.468585 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731c35f-a02e-47f0-bf64-b36d367351ea admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:49.473884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7be034-8506-4f3f-8549-d4541913716d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:49.474743 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7be034-8506-4f3f-8549-d4541913716d service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:49.475181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7be034-8506-4f3f-8549-d4541913716d service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:49.477081 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7be034-8506-4f3f-8549-d4541913716d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:49.493522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7be034-8506-4f3f-8549-d4541913716d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:49.495829 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7be034-8506-4f3f-8549-d4541913716d service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:49.514986 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1159/2370] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:34:50.750881 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cf004-37cf-4459-a49f-a67118110246 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:50.751419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cf004-37cf-4459-a49f-a67118110246 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:50.751535 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cf004-37cf-4459-a49f-a67118110246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:50.812783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ad8cf004-37cf-4459-a49f-a67118110246 None None] MFA Rules not processed for user `068f28eb74e74650a2b80d29664109ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:50.818413 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ad8cf004-37cf-4459-a49f-a67118110246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:50.912437 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:51.004601 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1212/2371] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:51.256050 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7be034-8506-4f3f-8549-d4541913716d service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:51.258598 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-be7be034-8506-4f3f-8549-d4541913716d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:51.329530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7be034-8506-4f3f-8549-d4541913716d service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:51.370353 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c9b36-96e8-4328-b8c0-28e4c697650e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:51.372161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c9b36-96e8-4328-b8c0-28e4c697650e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:51.373396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c9b36-96e8-4328-b8c0-28e4c697650e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:51.373854 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c9b36-96e8-4328-b8c0-28e4c697650e admin admin] PATH_INFO: `/v3/projects/88ef468ef8294bcab8d4c98a9ab3bd86` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:51.375821 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c9b36-96e8-4328-b8c0-28e4c697650e admin admin] RBAC: Authorizing `identity:delete_project(project_id=88ef468ef8294bcab8d4c98a9ab3bd86)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:51.378783 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c9b36-96e8-4328-b8c0-28e4c697650e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:51.412118 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:51.488413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3e1c9b36-96e8-4328-b8c0-28e4c697650e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88ef468ef8294bcab8d4c98a9ab3bd86', 'request_id': 'req-3e1c9b36-96e8-4328-b8c0-28e4c697650e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:51.499426 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3e1c9b36-96e8-4328-b8c0-28e4c697650e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88ef468ef8294bcab8d4c98a9ab3bd86', 'request_id': 'req-3e1c9b36-96e8-4328-b8c0-28e4c697650e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:51.514193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3e1c9b36-96e8-4328-b8c0-28e4c697650e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88ef468ef8294bcab8d4c98a9ab3bd86', 'request_id': 'req-3e1c9b36-96e8-4328-b8c0-28e4c697650e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:51.545336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3e1c9b36-96e8-4328-b8c0-28e4c697650e admin admin] The token cache is being invalidate because project 88ef468ef8294bcab8d4c98a9ab3bd86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:51.545736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3e1c9b36-96e8-4328-b8c0-28e4c697650e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:51.547556 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3e1c9b36-96e8-4328-b8c0-28e4c697650e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:51.549736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1160/2372] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:51 2022] DELETE /identity/v3/projects/88ef468ef8294bcab8d4c98a9ab3bd86 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:51.554655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b927815-219a-4da7-9574-7340c7fc0f77 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:51.554942 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b927815-219a-4da7-9574-7340c7fc0f77 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:51.555223 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b927815-219a-4da7-9574-7340c7fc0f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:51.600692 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-0b927815-219a-4da7-9574-7340c7fc0f77 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:51.601972 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0b927815-219a-4da7-9574-7340c7fc0f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:51.802880 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1213/2373] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:51.813774 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3f4c4-21b8-4ce9-a4ff-e38988d9dc7b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:51.814341 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3f4c4-21b8-4ce9-a4ff-e38988d9dc7b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:51.814777 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3f4c4-21b8-4ce9-a4ff-e38988d9dc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:51.881970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d5d3f4c4-21b8-4ce9-a4ff-e38988d9dc7b None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:51.889134 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d5d3f4c4-21b8-4ce9-a4ff-e38988d9dc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:51.934918 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1161/2374] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:51.936954 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b927815-219a-4da7-9574-7340c7fc0f77 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:51.953839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b927815-219a-4da7-9574-7340c7fc0f77 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:51.976734 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca6cdd-26f3-4726-8c1b-768acb3afa01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:51.977499 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca6cdd-26f3-4726-8c1b-768acb3afa01 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:51.977713 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca6cdd-26f3-4726-8c1b-768acb3afa01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:51.977993 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca6cdd-26f3-4726-8c1b-768acb3afa01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:51.979214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca6cdd-26f3-4726-8c1b-768acb3afa01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:34:51.979522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca6cdd-26f3-4726-8c1b-768acb3afa01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:51.989808 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca6cdd-26f3-4726-8c1b-768acb3afa01 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:52.033684 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1214/2375] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:34:52.041243 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3f4c4-21b8-4ce9-a4ff-e38988d9dc7b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:52.073916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3f4c4-21b8-4ce9-a4ff-e38988d9dc7b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:52.090806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac5837-c3ff-49e8-823b-e2adfa3789a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:52.091113 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac5837-c3ff-49e8-823b-e2adfa3789a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:52.091303 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac5837-c3ff-49e8-823b-e2adfa3789a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:52.091589 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac5837-c3ff-49e8-823b-e2adfa3789a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:52.092258 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac5837-c3ff-49e8-823b-e2adfa3789a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:52.096425 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac5837-c3ff-49e8-823b-e2adfa3789a1 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:52.159193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1162/2376] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:52 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:52.164887 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca6cdd-26f3-4726-8c1b-768acb3afa01 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:52.190058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca6cdd-26f3-4726-8c1b-768acb3afa01 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:52.411789 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa89f92-ec42-42b3-8273-94e454600d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:52.412809 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa89f92-ec42-42b3-8273-94e454600d1f admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:52.413248 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa89f92-ec42-42b3-8273-94e454600d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:52.414028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa89f92-ec42-42b3-8273-94e454600d1f admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:52.415034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa89f92-ec42-42b3-8273-94e454600d1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:52.418331 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa89f92-ec42-42b3-8273-94e454600d1f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:52.449751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1215/2377] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:52 2022] POST /identity/v3/users => generated 286 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:52.461544 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac5837-c3ff-49e8-823b-e2adfa3789a1 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:52.484914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac5837-c3ff-49e8-823b-e2adfa3789a1 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:52.499531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2813c6f-18b2-4120-83a7-e18a6f48e463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:52.500319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2813c6f-18b2-4120-83a7-e18a6f48e463 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:52.500611 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2813c6f-18b2-4120-83a7-e18a6f48e463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:52.501039 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2813c6f-18b2-4120-83a7-e18a6f48e463 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:52.504808 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2813c6f-18b2-4120-83a7-e18a6f48e463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:52.512429 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2813c6f-18b2-4120-83a7-e18a6f48e463 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:52.524867 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1163/2378] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:52.529051 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa89f92-ec42-42b3-8273-94e454600d1f admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:52.543526 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa89f92-ec42-42b3-8273-94e454600d1f admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:52.551772 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4de1aa-f0bf-4871-b340-705abe4bfbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:52.553367 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4de1aa-f0bf-4871-b340-705abe4bfbf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:52.554167 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4de1aa-f0bf-4871-b340-705abe4bfbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:52.554705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4de1aa-f0bf-4871-b340-705abe4bfbf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:52.555684 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4de1aa-f0bf-4871-b340-705abe4bfbf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:52.560050 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4de1aa-f0bf-4871-b340-705abe4bfbf9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:52.580986 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1216/2379] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:34:52.585516 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2813c6f-18b2-4120-83a7-e18a6f48e463 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:52.599529 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2813c6f-18b2-4120-83a7-e18a6f48e463 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:52.607455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199fc2b-b0f2-446a-832d-930c4e01906a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:52.615071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199fc2b-b0f2-446a-832d-930c4e01906a admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:52.615603 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199fc2b-b0f2-446a-832d-930c4e01906a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:52.616070 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199fc2b-b0f2-446a-832d-930c4e01906a admin admin] PATH_INFO: `/v3/projects/2f93e5b4f52b4b8cac7e0336a44b33ba/users/b93ff419d60043f0aab0c11a6abdaa38/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:52.657495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199fc2b-b0f2-446a-832d-930c4e01906a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f93e5b4f52b4b8cac7e0336a44b33ba, user_id=b93ff419d60043f0aab0c11a6abdaa38, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:52.668321 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199fc2b-b0f2-446a-832d-930c4e01906a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:52.696622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1164/2380] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:52 2022] PUT /identity/v3/projects/2f93e5b4f52b4b8cac7e0336a44b33ba/users/b93ff419d60043f0aab0c11a6abdaa38/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:52.702708 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb24b84-767f-4675-bc87-11928035c80d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:52.703385 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb24b84-767f-4675-bc87-11928035c80d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:52.704118 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb24b84-767f-4675-bc87-11928035c80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:52.734709 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ceb24b84-767f-4675-bc87-11928035c80d None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:52.739657 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ceb24b84-767f-4675-bc87-11928035c80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:52.800602 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:52.850422 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1217/2381] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:52.880806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199fc2b-b0f2-446a-832d-930c4e01906a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:52.883033 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2199fc2b-b0f2-446a-832d-930c4e01906a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:53.080524 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb24b84-767f-4675-bc87-11928035c80d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:53.082356 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ceb24b84-767f-4675-bc87-11928035c80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:53.106423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb24b84-767f-4675-bc87-11928035c80d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:53.140834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc8478-a449-41a8-b5f2-127e9f0536c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:53.141494 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc8478-a449-41a8-b5f2-127e9f0536c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:53.141769 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc8478-a449-41a8-b5f2-127e9f0536c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:53.142216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc8478-a449-41a8-b5f2-127e9f0536c7 admin admin] PATH_INFO: `/v3/users/34e80e67c7d34441966451c0de3c2a2e` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:53.143699 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc8478-a449-41a8-b5f2-127e9f0536c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34e80e67c7d34441966451c0de3c2a2e)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:53.146910 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc8478-a449-41a8-b5f2-127e9f0536c7 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:53.174298 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199fc2b-b0f2-446a-832d-930c4e01906a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:53.190553 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adcff19-4399-44ac-8e3d-561ce314da73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:53.191207 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcff19-4399-44ac-8e3d-561ce314da73 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:53.191485 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcff19-4399-44ac-8e3d-561ce314da73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:53.191756 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcff19-4399-44ac-8e3d-561ce314da73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:53.221671 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adcff19-4399-44ac-8e3d-561ce314da73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:53.223147 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adcff19-4399-44ac-8e3d-561ce314da73 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:53.283718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1165/2382] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:34:53.338084 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-55fc8478-a449-41a8-b5f2-127e9f0536c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34e80e67c7d34441966451c0de3c2a2e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:53.354822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-55fc8478-a449-41a8-b5f2-127e9f0536c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34e80e67c7d34441966451c0de3c2a2e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:53.373580 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-55fc8478-a449-41a8-b5f2-127e9f0536c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34e80e67c7d34441966451c0de3c2a2e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:53.374360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-55fc8478-a449-41a8-b5f2-127e9f0536c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34e80e67c7d34441966451c0de3c2a2e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:53.375497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-55fc8478-a449-41a8-b5f2-127e9f0536c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34e80e67c7d34441966451c0de3c2a2e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:53.429522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1218/2383] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:53 2022] DELETE /identity/v3/users/34e80e67c7d34441966451c0de3c2a2e => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:34:53.432792 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529b238-19a5-45d4-b3c0-64a3da2fa355 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:53.433545 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529b238-19a5-45d4-b3c0-64a3da2fa355 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:53.434300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529b238-19a5-45d4-b3c0-64a3da2fa355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:53.502629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b529b238-19a5-45d4-b3c0-64a3da2fa355 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:53.505838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b529b238-19a5-45d4-b3c0-64a3da2fa355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:53.717317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:53.787034 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1166/2384] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:53.849944 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc8478-a449-41a8-b5f2-127e9f0536c7 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:53.852067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-55fc8478-a449-41a8-b5f2-127e9f0536c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:54.015101 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc8478-a449-41a8-b5f2-127e9f0536c7 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:54.022023 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfeeb8b-c732-4712-8b98-92fae6da1a93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:54.024910 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfeeb8b-c732-4712-8b98-92fae6da1a93 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:54.025528 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfeeb8b-c732-4712-8b98-92fae6da1a93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:54.026184 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfeeb8b-c732-4712-8b98-92fae6da1a93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:54.066702 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfeeb8b-c732-4712-8b98-92fae6da1a93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:54.072579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfeeb8b-c732-4712-8b98-92fae6da1a93 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:54.099010 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1219/2385] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:56.962115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529b238-19a5-45d4-b3c0-64a3da2fa355 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:56.967579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b529b238-19a5-45d4-b3c0-64a3da2fa355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:57.063123 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529b238-19a5-45d4-b3c0-64a3da2fa355 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:57.083512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d89178-0798-4f9e-87ec-154fb7a9f348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:57.084974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d89178-0798-4f9e-87ec-154fb7a9f348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:57.085633 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d89178-0798-4f9e-87ec-154fb7a9f348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:57.086296 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d89178-0798-4f9e-87ec-154fb7a9f348 admin admin] PATH_INFO: `/v3/projects/7f91610898f74be3b832cc55807e4a7d` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:57.095005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d89178-0798-4f9e-87ec-154fb7a9f348 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f91610898f74be3b832cc55807e4a7d)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:57.109041 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d89178-0798-4f9e-87ec-154fb7a9f348 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:57.244613 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfeeb8b-c732-4712-8b98-92fae6da1a93 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:57.247312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9bfeeb8b-c732-4712-8b98-92fae6da1a93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:57.308228 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfeeb8b-c732-4712-8b98-92fae6da1a93 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:57.322580 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-93d89178-0798-4f9e-87ec-154fb7a9f348 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f91610898f74be3b832cc55807e4a7d', 'request_id': 'req-93d89178-0798-4f9e-87ec-154fb7a9f348'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:57.334715 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-93d89178-0798-4f9e-87ec-154fb7a9f348 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f91610898f74be3b832cc55807e4a7d', 'request_id': 'req-93d89178-0798-4f9e-87ec-154fb7a9f348'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:57.351313 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-93d89178-0798-4f9e-87ec-154fb7a9f348 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f91610898f74be3b832cc55807e4a7d', 'request_id': 'req-93d89178-0798-4f9e-87ec-154fb7a9f348'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:57.373761 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-93d89178-0798-4f9e-87ec-154fb7a9f348 admin admin] The token cache is being invalidate because project 7f91610898f74be3b832cc55807e4a7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:34:57.376721 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-93d89178-0798-4f9e-87ec-154fb7a9f348 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:57.377480 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-93d89178-0798-4f9e-87ec-154fb7a9f348 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:57.380478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1167/2386] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:34:56 2022] DELETE /identity/v3/projects/7f91610898f74be3b832cc55807e4a7d => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:34:57.385061 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973b4b8-2000-4158-8103-c86435494188 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:57.385061 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973b4b8-2000-4158-8103-c86435494188 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:57.385413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973b4b8-2000-4158-8103-c86435494188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:57.414301 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6973b4b8-2000-4158-8103-c86435494188 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:57.415575 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6973b4b8-2000-4158-8103-c86435494188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:57.434527 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:57.448775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592a9dd-902b-4226-8075-5a993e0bf636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:57.450786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592a9dd-902b-4226-8075-5a993e0bf636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:57.451389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592a9dd-902b-4226-8075-5a993e0bf636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:57.452102 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592a9dd-902b-4226-8075-5a993e0bf636 admin admin] PATH_INFO: `/v3/users/dc5e45f9539048c4a85a649d486d1ba3` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:57.454341 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c592a9dd-902b-4226-8075-5a993e0bf636 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc5e45f9539048c4a85a649d486d1ba3)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:57.457313 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c592a9dd-902b-4226-8075-5a993e0bf636 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:57.533099 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1168/2387] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:57.535678 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b63c5-c6d3-42d7-802d-764e634d61b9 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:57.537090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b63c5-c6d3-42d7-802d-764e634d61b9 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:57.538076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b63c5-c6d3-42d7-802d-764e634d61b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:57.557568 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c592a9dd-902b-4226-8075-5a993e0bf636 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc5e45f9539048c4a85a649d486d1ba3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:57.594487 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-9e4b63c5-c6d3-42d7-802d-764e634d61b9 None None] MFA Rules not processed for user `4c5ce3754d6d4af8a325683564c48538`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:57.596406 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9e4b63c5-c6d3-42d7-802d-764e634d61b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:57.599626 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c592a9dd-902b-4226-8075-5a993e0bf636 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc5e45f9539048c4a85a649d486d1ba3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:57.614670 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c592a9dd-902b-4226-8075-5a993e0bf636 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc5e45f9539048c4a85a649d486d1ba3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:57.616286 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c592a9dd-902b-4226-8075-5a993e0bf636 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc5e45f9539048c4a85a649d486d1ba3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:57.618092 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c592a9dd-902b-4226-8075-5a993e0bf636 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc5e45f9539048c4a85a649d486d1ba3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:34:57.661210 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1220/2388] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:34:57 2022] DELETE /identity/v3/users/dc5e45f9539048c4a85a649d486d1ba3 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:34:57.666287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a981fc4-d1b9-4573-8efc-8cefdea9edea None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:57.668776 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a981fc4-d1b9-4573-8efc-8cefdea9edea None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:57.669076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a981fc4-d1b9-4573-8efc-8cefdea9edea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:57.701002 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:57.719968 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-5a981fc4-d1b9-4573-8efc-8cefdea9edea None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:57.724889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5a981fc4-d1b9-4573-8efc-8cefdea9edea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:57.738586 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1169/2389] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:57.742054 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304b863-439d-4860-b0e6-11616f8b4dc4 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:57.742528 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304b863-439d-4860-b0e6-11616f8b4dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:57.742714 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304b863-439d-4860-b0e6-11616f8b4dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:57.793549 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-2304b863-439d-4860-b0e6-11616f8b4dc4 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:57.796129 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2304b863-439d-4860-b0e6-11616f8b4dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:57.828852 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1221/2390] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:57.832491 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a981fc4-d1b9-4573-8efc-8cefdea9edea None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:57.835019 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5a981fc4-d1b9-4573-8efc-8cefdea9edea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:57.904649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1170/2391] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:57.906032 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304b863-439d-4860-b0e6-11616f8b4dc4 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:57.941733 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304b863-439d-4860-b0e6-11616f8b4dc4 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:57.969993 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99daca-a0b7-4c53-a23d-a5f966ffd1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:57.970851 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99daca-a0b7-4c53-a23d-a5f966ffd1a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:57.971039 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99daca-a0b7-4c53-a23d-a5f966ffd1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:57.971310 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99daca-a0b7-4c53-a23d-a5f966ffd1a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:57.973181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99daca-a0b7-4c53-a23d-a5f966ffd1a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:34:57.973655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99daca-a0b7-4c53-a23d-a5f966ffd1a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:57.976944 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99daca-a0b7-4c53-a23d-a5f966ffd1a0 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:58.014911 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1171/2392] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:34:58.015923 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99daca-a0b7-4c53-a23d-a5f966ffd1a0 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:58.025867 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ac99daca-a0b7-4c53-a23d-a5f966ffd1a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:58.058764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a981fc4-d1b9-4573-8efc-8cefdea9edea None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:58.068249 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d1bd3-29f8-4fa9-ad44-0054f45bdda4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d022ed8d2ac43ae816aa396a8e2f17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:58.068891 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d1bd3-29f8-4fa9-ad44-0054f45bdda4 service glance] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:58.069332 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d1bd3-29f8-4fa9-ad44-0054f45bdda4 service glance] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:58.069455 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d1bd3-29f8-4fa9-ad44-0054f45bdda4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:58.071850 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c85d1bd3-29f8-4fa9-ad44-0054f45bdda4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:58.105560 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:58.190239 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99daca-a0b7-4c53-a23d-a5f966ffd1a0 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:58.199877 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc97d6-b567-4de5-a150-30f875d46409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:58.200552 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc97d6-b567-4de5-a150-30f875d46409 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:58.200836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc97d6-b567-4de5-a150-30f875d46409 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:58.201111 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc97d6-b567-4de5-a150-30f875d46409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:58.214257 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc97d6-b567-4de5-a150-30f875d46409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:58.215090 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d1bd3-29f8-4fa9-ad44-0054f45bdda4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:58.215637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc97d6-b567-4de5-a150-30f875d46409 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:58.217437 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d1bd3-29f8-4fa9-ad44-0054f45bdda4 service glance] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:58.231157 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1172/2393] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:58.232120 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc97d6-b567-4de5-a150-30f875d46409 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:58.248397 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc97d6-b567-4de5-a150-30f875d46409 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:58.253521 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3d376-a6ff-41f9-8b11-7f09ffa2a2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:58.254142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3d376-a6ff-41f9-8b11-7f09ffa2a2a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:58.255171 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3d376-a6ff-41f9-8b11-7f09ffa2a2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:58.255171 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3d376-a6ff-41f9-8b11-7f09ffa2a2a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:58.255482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3d376-a6ff-41f9-8b11-7f09ffa2a2a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:58.257312 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3d376-a6ff-41f9-8b11-7f09ffa2a2a2 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:58.262861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1222/2394] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:34:58.366784 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1173/2395] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:34:58 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:58.371494 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d1bd3-29f8-4fa9-ad44-0054f45bdda4 service glance] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:58.400721 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d1bd3-29f8-4fa9-ad44-0054f45bdda4 service glance] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:58.710464 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb4211-b5cb-4cd8-8fe8-45532ae82eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:58.711696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb4211-b5cb-4cd8-8fe8-45532ae82eb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:58.716451 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb4211-b5cb-4cd8-8fe8-45532ae82eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:58.716942 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbb4211-b5cb-4cd8-8fe8-45532ae82eb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:58.721156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbb4211-b5cb-4cd8-8fe8-45532ae82eb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:58.728852 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbb4211-b5cb-4cd8-8fe8-45532ae82eb2 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:58.748443 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444cd9e-2d28-448e-8daf-4c29774050d6 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:58.748718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444cd9e-2d28-448e-8daf-4c29774050d6 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:58.749065 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444cd9e-2d28-448e-8daf-4c29774050d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:58.798001 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1223/2396] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:34:58 2022] POST /identity/v3/users => generated 285 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:34:58.801279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-4444cd9e-2d28-448e-8daf-4c29774050d6 None None] MFA Rules not processed for user `61709ccc1e244185b252179451cd860f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:58.802570 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4444cd9e-2d28-448e-8daf-4c29774050d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:58.805472 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb4211-b5cb-4cd8-8fe8-45532ae82eb2 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:58.822722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbb4211-b5cb-4cd8-8fe8-45532ae82eb2 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:58.831520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bfe886-d584-4d92-a49c-dba2323740df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:58.833236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfe886-d584-4d92-a49c-dba2323740df admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:58.833867 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfe886-d584-4d92-a49c-dba2323740df admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:58.834559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfe886-d584-4d92-a49c-dba2323740df admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:58.835594 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bfe886-d584-4d92-a49c-dba2323740df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:58.839677 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bfe886-d584-4d92-a49c-dba2323740df admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:58.862463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1224/2397] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:58.866821 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bfe886-d584-4d92-a49c-dba2323740df admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:58.896743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bfe886-d584-4d92-a49c-dba2323740df admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:58.911644 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effb5f2-7df1-4cc7-8cba-7dc75af6ff0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:58.917589 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effb5f2-7df1-4cc7-8cba-7dc75af6ff0b admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:58.918753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effb5f2-7df1-4cc7-8cba-7dc75af6ff0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:58.919426 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3effb5f2-7df1-4cc7-8cba-7dc75af6ff0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:58.922656 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3effb5f2-7df1-4cc7-8cba-7dc75af6ff0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:58.928258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3effb5f2-7df1-4cc7-8cba-7dc75af6ff0b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:58.946393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1174/2398] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 29 20:34:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 29 20:34:58.981657 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1225/2399] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:34:58.986901 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444cd9e-2d28-448e-8daf-4c29774050d6 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:58.999251 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:59.008188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444cd9e-2d28-448e-8daf-4c29774050d6 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:59.025862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dab886-c2f7-4044-ba77-0a8a49383e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:59.027100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dab886-c2f7-4044-ba77-0a8a49383e6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:59.027583 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dab886-c2f7-4044-ba77-0a8a49383e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:59.028197 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dab886-c2f7-4044-ba77-0a8a49383e6c admin admin] PATH_INFO: `/v3/projects/1c85b91350b74daeab5ba89364b4578e/users/93dd431edc40415295518d9d24fd94af/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:59.047052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effb5f2-7df1-4cc7-8cba-7dc75af6ff0b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:59.049323 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3effb5f2-7df1-4cc7-8cba-7dc75af6ff0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:59.085810 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dab886-c2f7-4044-ba77-0a8a49383e6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c85b91350b74daeab5ba89364b4578e, user_id=93dd431edc40415295518d9d24fd94af, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:59.100338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dab886-c2f7-4044-ba77-0a8a49383e6c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:59.137176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1175/2400] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:34:58 2022] PUT /identity/v3/projects/1c85b91350b74daeab5ba89364b4578e/users/93dd431edc40415295518d9d24fd94af/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:34:59.142154 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50d4c6-a130-4830-9a73-2f748896aeb6 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:59.142839 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50d4c6-a130-4830-9a73-2f748896aeb6 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:59.145410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50d4c6-a130-4830-9a73-2f748896aeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:59.211372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-8a50d4c6-a130-4830-9a73-2f748896aeb6 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:34:59.212657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8a50d4c6-a130-4830-9a73-2f748896aeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:34:59.274360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3effb5f2-7df1-4cc7-8cba-7dc75af6ff0b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:59.343996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:34:59.402313 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1176/2401] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:34:59.423739 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd4a2f-c7b9-4777-bc21-f3caf1493efe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599129b7d8ff47efb145787414ddb80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:59.425023 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd4a2f-c7b9-4777-bc21-f3caf1493efe service swift] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:59.425552 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd4a2f-c7b9-4777-bc21-f3caf1493efe service swift] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:59.425911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd4a2f-c7b9-4777-bc21-f3caf1493efe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:59.442938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50d4c6-a130-4830-9a73-2f748896aeb6 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:34:59.542458 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50d4c6-a130-4830-9a73-2f748896aeb6 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:34:59.548264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16e511-182e-4de5-b9a9-7f796c6ec33c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:34:59.548949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16e511-182e-4de5-b9a9-7f796c6ec33c service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:34:59.549291 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16e511-182e-4de5-b9a9-7f796c6ec33c service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:34:59.549983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16e511-182e-4de5-b9a9-7f796c6ec33c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:34:59.583232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16e511-182e-4de5-b9a9-7f796c6ec33c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:59.584864 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d16e511-182e-4de5-b9a9-7f796c6ec33c service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:59.587059 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd4a2f-c7b9-4777-bc21-f3caf1493efe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:34:59.589880 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd4a2f-c7b9-4777-bc21-f3caf1493efe service swift] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:34:59.611331 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1177/2402] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:34:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:34:59.625758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1226/2403] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 29 20:34:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:35:01.416526 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16e511-182e-4de5-b9a9-7f796c6ec33c service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:01.420979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-3d16e511-182e-4de5-b9a9-7f796c6ec33c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:01.478598 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16e511-182e-4de5-b9a9-7f796c6ec33c service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:01.536991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c53ff-fdb2-427c-9505-9b5089ee5025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:01.563261 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c53ff-fdb2-427c-9505-9b5089ee5025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:01.565658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c53ff-fdb2-427c-9505-9b5089ee5025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:01.566316 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c53ff-fdb2-427c-9505-9b5089ee5025 admin admin] PATH_INFO: `/v3/projects/d26c88c225c542eaa1cfb2c459e292f6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:01.568484 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c53ff-fdb2-427c-9505-9b5089ee5025 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d26c88c225c542eaa1cfb2c459e292f6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:01.571381 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c53ff-fdb2-427c-9505-9b5089ee5025 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:01.810943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ee6c53ff-fdb2-427c-9505-9b5089ee5025 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd26c88c225c542eaa1cfb2c459e292f6', 'request_id': 'req-ee6c53ff-fdb2-427c-9505-9b5089ee5025'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:01.833355 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ee6c53ff-fdb2-427c-9505-9b5089ee5025 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd26c88c225c542eaa1cfb2c459e292f6', 'request_id': 'req-ee6c53ff-fdb2-427c-9505-9b5089ee5025'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:01.859522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ee6c53ff-fdb2-427c-9505-9b5089ee5025 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd26c88c225c542eaa1cfb2c459e292f6', 'request_id': 'req-ee6c53ff-fdb2-427c-9505-9b5089ee5025'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:01.891445 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ee6c53ff-fdb2-427c-9505-9b5089ee5025 admin admin] The token cache is being invalidate because project d26c88c225c542eaa1cfb2c459e292f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:35:01.891813 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ee6c53ff-fdb2-427c-9505-9b5089ee5025 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:01.918410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ee6c53ff-fdb2-427c-9505-9b5089ee5025 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:01.933560 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1178/2404] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:35:01 2022] DELETE /identity/v3/projects/d26c88c225c542eaa1cfb2c459e292f6 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:01.936745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034c387-9d10-48f5-9e1f-e3224045ff3f None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:01.937339 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034c387-9d10-48f5-9e1f-e3224045ff3f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:01.937504 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034c387-9d10-48f5-9e1f-e3224045ff3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:01.985029 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a034c387-9d10-48f5-9e1f-e3224045ff3f None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:01.986739 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a034c387-9d10-48f5-9e1f-e3224045ff3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:02.149931 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1227/2405] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:35:02.153386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edf8a5-e53e-48f3-bfac-f12990318070 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:02.153999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edf8a5-e53e-48f3-bfac-f12990318070 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:02.154522 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62edf8a5-e53e-48f3-bfac-f12990318070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:02.194951 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:02.207471 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-62edf8a5-e53e-48f3-bfac-f12990318070 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:02.213666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-62edf8a5-e53e-48f3-bfac-f12990318070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:02.241550 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1179/2406] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:02.242650 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034c387-9d10-48f5-9e1f-e3224045ff3f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:02.256245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034c387-9d10-48f5-9e1f-e3224045ff3f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:02.269396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b511a3d-63fc-43a3-b888-99b9fcd4a697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:02.270623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b511a3d-63fc-43a3-b888-99b9fcd4a697 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:02.271009 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b511a3d-63fc-43a3-b888-99b9fcd4a697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:02.272129 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b511a3d-63fc-43a3-b888-99b9fcd4a697 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:02.274538 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b511a3d-63fc-43a3-b888-99b9fcd4a697 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:35:02.274950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b511a3d-63fc-43a3-b888-99b9fcd4a697 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:02.279823 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b511a3d-63fc-43a3-b888-99b9fcd4a697 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:02.321073 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1228/2407] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:35:02.322983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62edf8a5-e53e-48f3-bfac-f12990318070 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:02.335600 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62edf8a5-e53e-48f3-bfac-f12990318070 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:02.349441 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd6204-7db4-4377-8ab4-5488161ae25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:02.350232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd6204-7db4-4377-8ab4-5488161ae25d admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:02.350606 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd6204-7db4-4377-8ab4-5488161ae25d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:02.350974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fd6204-7db4-4377-8ab4-5488161ae25d admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:02.351927 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd6204-7db4-4377-8ab4-5488161ae25d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:02.354736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fd6204-7db4-4377-8ab4-5488161ae25d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:02.438246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1180/2408] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:35:02 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:35:02.442495 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b511a3d-63fc-43a3-b888-99b9fcd4a697 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:02.471612 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b511a3d-63fc-43a3-b888-99b9fcd4a697 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:02.652264 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:02.688529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598980e-a663-418a-93e1-53bacb90f63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:02.689379 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598980e-a663-418a-93e1-53bacb90f63b admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:02.689712 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598980e-a663-418a-93e1-53bacb90f63b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:02.689992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598980e-a663-418a-93e1-53bacb90f63b admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:02.706298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6598980e-a663-418a-93e1-53bacb90f63b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:02.709646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6598980e-a663-418a-93e1-53bacb90f63b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:02.781208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1229/2409] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:35:02 2022] POST /identity/v3/users => generated 287 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:35:02.785832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd6204-7db4-4377-8ab4-5488161ae25d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:02.802344 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fd6204-7db4-4377-8ab4-5488161ae25d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:02.827276 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a95483f-436e-4360-96c4-2af8ea559d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:02.828140 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95483f-436e-4360-96c4-2af8ea559d30 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:02.828531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95483f-436e-4360-96c4-2af8ea559d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:02.828905 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a95483f-436e-4360-96c4-2af8ea559d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:02.829850 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a95483f-436e-4360-96c4-2af8ea559d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:02.833098 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a95483f-436e-4360-96c4-2af8ea559d30 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:02.864911 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1181/2410] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:35:02.870275 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598980e-a663-418a-93e1-53bacb90f63b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:02.889600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598980e-a663-418a-93e1-53bacb90f63b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:02.897438 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1fa4d6-1691-44df-87a0-cf0c8ceb09a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:02.898615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fa4d6-1691-44df-87a0-cf0c8ceb09a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:02.898910 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fa4d6-1691-44df-87a0-cf0c8ceb09a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:02.899238 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fa4d6-1691-44df-87a0-cf0c8ceb09a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:02.899913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1fa4d6-1691-44df-87a0-cf0c8ceb09a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:02.901932 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1fa4d6-1691-44df-87a0-cf0c8ceb09a6 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:02.928707 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1230/2411] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:35:02.934350 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a95483f-436e-4360-96c4-2af8ea559d30 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:02.973253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a95483f-436e-4360-96c4-2af8ea559d30 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:03.049244 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4b7fb-6c82-475c-ab0d-1c1e3be93abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:03.050348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4b7fb-6c82-475c-ab0d-1c1e3be93abd admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:03.050804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4b7fb-6c82-475c-ab0d-1c1e3be93abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:03.051271 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4b7fb-6c82-475c-ab0d-1c1e3be93abd admin admin] PATH_INFO: `/v3/projects/b1d029424ba747d6a185dd0074df8e1f/users/67976416c9104dc58849907db0c566ce/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:03.137108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4b7fb-6c82-475c-ab0d-1c1e3be93abd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1d029424ba747d6a185dd0074df8e1f, user_id=67976416c9104dc58849907db0c566ce, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:03.139803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4b7fb-6c82-475c-ab0d-1c1e3be93abd admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:03.194155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1182/2412] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:35:02 2022] PUT /identity/v3/projects/b1d029424ba747d6a185dd0074df8e1f/users/67976416c9104dc58849907db0c566ce/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:03.313301 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5901c3-7b61-45d1-9101-257b5e505d5a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:03.313713 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5901c3-7b61-45d1-9101-257b5e505d5a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:03.313911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5901c3-7b61-45d1-9101-257b5e505d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:03.403005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ce5901c3-7b61-45d1-9101-257b5e505d5a None None] MFA Rules not processed for user `67976416c9104dc58849907db0c566ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:03.405134 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ce5901c3-7b61-45d1-9101-257b5e505d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:03.578329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1231/2413] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:35:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:03.640771 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4b7fb-6c82-475c-ab0d-1c1e3be93abd admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:03.642881 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a2d4b7fb-6c82-475c-ab0d-1c1e3be93abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:03.974155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4b7fb-6c82-475c-ab0d-1c1e3be93abd admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:03.989020 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a109e2-1a59-4c9d-977a-a30d4ea28776 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:03.990422 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a109e2-1a59-4c9d-977a-a30d4ea28776 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:03.991133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a109e2-1a59-4c9d-977a-a30d4ea28776 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:03.991817 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a109e2-1a59-4c9d-977a-a30d4ea28776 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:04.042149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a109e2-1a59-4c9d-977a-a30d4ea28776 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:04.053058 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a109e2-1a59-4c9d-977a-a30d4ea28776 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:04.102113 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1183/2414] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:35:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:35:04.167443 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5901c3-7b61-45d1-9101-257b5e505d5a None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:04.372620 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5901c3-7b61-45d1-9101-257b5e505d5a None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:04.388014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce4288-1b30-47a0-b79a-0fc8fedf89d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:04.388537 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce4288-1b30-47a0-b79a-0fc8fedf89d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:04.388852 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce4288-1b30-47a0-b79a-0fc8fedf89d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:04.389135 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce4288-1b30-47a0-b79a-0fc8fedf89d9 admin admin] PATH_INFO: `/v3/users/67976416c9104dc58849907db0c566ce` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:04.391390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce4288-1b30-47a0-b79a-0fc8fedf89d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67976416c9104dc58849907db0c566ce)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:04.397305 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce4288-1b30-47a0-b79a-0fc8fedf89d9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:04.492913 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f939a84-7726-4f54-9654-cf7422ed57dd None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:04.493409 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f939a84-7726-4f54-9654-cf7422ed57dd None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:04.495302 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f939a84-7726-4f54-9654-cf7422ed57dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:04.623082 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5f939a84-7726-4f54-9654-cf7422ed57dd None None] MFA Rules not processed for user `93dd431edc40415295518d9d24fd94af`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:04.629571 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5f939a84-7726-4f54-9654-cf7422ed57dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:04.659761 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-32ce4288-1b30-47a0-b79a-0fc8fedf89d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67976416c9104dc58849907db0c566ce'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:04.712127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-32ce4288-1b30-47a0-b79a-0fc8fedf89d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67976416c9104dc58849907db0c566ce'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:04.730926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:04.766061 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-32ce4288-1b30-47a0-b79a-0fc8fedf89d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67976416c9104dc58849907db0c566ce'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:04.768609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-32ce4288-1b30-47a0-b79a-0fc8fedf89d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67976416c9104dc58849907db0c566ce'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:04.769830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-32ce4288-1b30-47a0-b79a-0fc8fedf89d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67976416c9104dc58849907db0c566ce'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:04.805865 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1232/2415] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:35:04 2022] DELETE /identity/v3/users/67976416c9104dc58849907db0c566ce => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:04.809381 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794f199-03d2-48cf-8fec-2fdfccd3d040 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:04.810224 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794f199-03d2-48cf-8fec-2fdfccd3d040 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:04.810693 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794f199-03d2-48cf-8fec-2fdfccd3d040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:04.857238 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-c794f199-03d2-48cf-8fec-2fdfccd3d040 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:04.858743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c794f199-03d2-48cf-8fec-2fdfccd3d040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:04.941625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1184/2416] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:35:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:04.947439 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f939a84-7726-4f54-9654-cf7422ed57dd None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:04.959081 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5f939a84-7726-4f54-9654-cf7422ed57dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:05.025544 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f939a84-7726-4f54-9654-cf7422ed57dd None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:05.056362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1233/2417] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:35:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:05.095967 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794f199-03d2-48cf-8fec-2fdfccd3d040 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:05.098327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c794f199-03d2-48cf-8fec-2fdfccd3d040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:05.142379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:05.198868 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593f64b-9790-4c8a-be6f-4d7788fc22dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:05.199663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593f64b-9790-4c8a-be6f-4d7788fc22dd service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:05.201961 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593f64b-9790-4c8a-be6f-4d7788fc22dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:05.202355 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593f64b-9790-4c8a-be6f-4d7788fc22dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:05.331394 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794f199-03d2-48cf-8fec-2fdfccd3d040 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:05.359452 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d8ea6-cb9a-4dd3-af2a-e3e898528bf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:05.360929 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d8ea6-cb9a-4dd3-af2a-e3e898528bf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:05.363264 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d8ea6-cb9a-4dd3-af2a-e3e898528bf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:05.364092 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d8ea6-cb9a-4dd3-af2a-e3e898528bf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:05.398935 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d8ea6-cb9a-4dd3-af2a-e3e898528bf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:05.401581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d8ea6-cb9a-4dd3-af2a-e3e898528bf8 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:05.430623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1234/2418] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:35:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:35:05.479181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1593f64b-9790-4c8a-be6f-4d7788fc22dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:05.480691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1593f64b-9790-4c8a-be6f-4d7788fc22dd service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:05.492391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1185/2419] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:35:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:35:06.689579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea730eae-9bc6-4c54-a5f8-ef380110164b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:06.690414 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea730eae-9bc6-4c54-a5f8-ef380110164b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:06.690832 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea730eae-9bc6-4c54-a5f8-ef380110164b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:06.759305 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ea730eae-9bc6-4c54-a5f8-ef380110164b None None] MFA Rules not processed for user `b93ff419d60043f0aab0c11a6abdaa38`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:06.761150 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ea730eae-9bc6-4c54-a5f8-ef380110164b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:06.961816 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1235/2420] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:35:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:35:06.981348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593f64b-9790-4c8a-be6f-4d7788fc22dd service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:06.983033 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1593f64b-9790-4c8a-be6f-4d7788fc22dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:07.035772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593f64b-9790-4c8a-be6f-4d7788fc22dd service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:07.085948 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4fc5b-e3ec-4260-9dfc-8cd2217fc347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:07.089076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4fc5b-e3ec-4260-9dfc-8cd2217fc347 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:07.089670 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4fc5b-e3ec-4260-9dfc-8cd2217fc347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:07.089792 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4fc5b-e3ec-4260-9dfc-8cd2217fc347 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:07.090955 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4fc5b-e3ec-4260-9dfc-8cd2217fc347 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:07.097586 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4fc5b-e3ec-4260-9dfc-8cd2217fc347 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:07.172330 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1186/2421] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:35:06 2022] POST /identity/v3/projects => generated 373 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:35:07.193998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea730eae-9bc6-4c54-a5f8-ef380110164b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:07.214800 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea730eae-9bc6-4c54-a5f8-ef380110164b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:07.228194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:07.407159 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf42e2-a325-4917-9842-3854cbbb0247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:07.407818 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf42e2-a325-4917-9842-3854cbbb0247 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:07.408135 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf42e2-a325-4917-9842-3854cbbb0247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:07.408420 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf42e2-a325-4917-9842-3854cbbb0247 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:07.409079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf42e2-a325-4917-9842-3854cbbb0247 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:07.415787 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf42e2-a325-4917-9842-3854cbbb0247 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:07.464097 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1236/2422] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:35:07 2022] POST /identity/v3/users => generated 293 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:35:07.469395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4fc5b-e3ec-4260-9dfc-8cd2217fc347 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:07.483613 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4fc5b-e3ec-4260-9dfc-8cd2217fc347 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:07.483835 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a9aeb-9a74-4ef4-942a-ecd5ab587f30 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:07.484135 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a9aeb-9a74-4ef4-942a-ecd5ab587f30 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:07.484418 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a9aeb-9a74-4ef4-942a-ecd5ab587f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:07.508725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245cebc-fed1-43c1-91a1-65fa7a56c3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:07.510284 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245cebc-fed1-43c1-91a1-65fa7a56c3dd admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:07.523650 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245cebc-fed1-43c1-91a1-65fa7a56c3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:07.524361 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245cebc-fed1-43c1-91a1-65fa7a56c3dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:07.526498 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245cebc-fed1-43c1-91a1-65fa7a56c3dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:07.533632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245cebc-fed1-43c1-91a1-65fa7a56c3dd admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:07.535770 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-1f0a9aeb-9a74-4ef4-942a-ecd5ab587f30 None None] MFA Rules not processed for user `93dd431edc40415295518d9d24fd94af`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:07.537922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1f0a9aeb-9a74-4ef4-942a-ecd5ab587f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:07.569325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1187/2423] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:35:07.570625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245cebc-fed1-43c1-91a1-65fa7a56c3dd admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:07.595906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245cebc-fed1-43c1-91a1-65fa7a56c3dd admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:07.609109 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab407efe-7395-4108-b0f3-64063fd4e64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:07.610433 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab407efe-7395-4108-b0f3-64063fd4e64d admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:07.611758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab407efe-7395-4108-b0f3-64063fd4e64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:07.615088 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab407efe-7395-4108-b0f3-64063fd4e64d admin admin] PATH_INFO: `/v3/projects/66bffbc642434ef2af36fae3e4a882a4/users/7f0b7046422a44e2acb959ad4878f079/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:07.663379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab407efe-7395-4108-b0f3-64063fd4e64d admin admin] RBAC: Authorizing `identity:create_grant(project_id=66bffbc642434ef2af36fae3e4a882a4, user_id=7f0b7046422a44e2acb959ad4878f079, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:07.667865 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab407efe-7395-4108-b0f3-64063fd4e64d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:07.695785 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1188/2424] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:35:07 2022] PUT /identity/v3/projects/66bffbc642434ef2af36fae3e4a882a4/users/7f0b7046422a44e2acb959ad4878f079/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:07.716833 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab407efe-7395-4108-b0f3-64063fd4e64d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:07.732127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1237/2425] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:35:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:35:07.901700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a9aeb-9a74-4ef4-942a-ecd5ab587f30 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:07.902392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab407efe-7395-4108-b0f3-64063fd4e64d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:07.904410 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1f0a9aeb-9a74-4ef4-942a-ecd5ab587f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:07.918043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f85b5-43e0-4b3c-9508-f5fa21ad15b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:07.918692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f85b5-43e0-4b3c-9508-f5fa21ad15b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:07.918974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f85b5-43e0-4b3c-9508-f5fa21ad15b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:07.919251 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f85b5-43e0-4b3c-9508-f5fa21ad15b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:07.920073 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f85b5-43e0-4b3c-9508-f5fa21ad15b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:07.928855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f85b5-43e0-4b3c-9508-f5fa21ad15b3 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:07.946486 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a9aeb-9a74-4ef4-942a-ecd5ab587f30 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:07.959071 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b29092-fb58-46ce-8378-996eab03722c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:07.959904 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b29092-fb58-46ce-8378-996eab03722c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:07.960579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b29092-fb58-46ce-8378-996eab03722c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:07.961555 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b29092-fb58-46ce-8378-996eab03722c admin admin] PATH_INFO: `/v3/projects/b1d029424ba747d6a185dd0074df8e1f` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:07.967369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b29092-fb58-46ce-8378-996eab03722c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1d029424ba747d6a185dd0074df8e1f)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:07.972701 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b29092-fb58-46ce-8378-996eab03722c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:07.986600 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1189/2426] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:35:07.997380 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f85b5-43e0-4b3c-9508-f5fa21ad15b3 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:08.026721 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f85b5-43e0-4b3c-9508-f5fa21ad15b3 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:08.049104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a499b-63f8-4fcd-9195-49fb69c13bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:08.050057 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a499b-63f8-4fcd-9195-49fb69c13bc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:08.057242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a499b-63f8-4fcd-9195-49fb69c13bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:08.057999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a499b-63f8-4fcd-9195-49fb69c13bc1 admin admin] PATH_INFO: `/v3/projects/66bffbc642434ef2af36fae3e4a882a4/users/7f0b7046422a44e2acb959ad4878f079/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:08.065127 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a499b-63f8-4fcd-9195-49fb69c13bc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66bffbc642434ef2af36fae3e4a882a4, user_id=7f0b7046422a44e2acb959ad4878f079, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:08.075578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a499b-63f8-4fcd-9195-49fb69c13bc1 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:08.111971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1190/2427] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:35:07 2022] PUT /identity/v3/projects/66bffbc642434ef2af36fae3e4a882a4/users/7f0b7046422a44e2acb959ad4878f079/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:08.123582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c2b29092-fb58-46ce-8378-996eab03722c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1d029424ba747d6a185dd0074df8e1f', 'request_id': 'req-c2b29092-fb58-46ce-8378-996eab03722c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:08.134965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c2b29092-fb58-46ce-8378-996eab03722c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1d029424ba747d6a185dd0074df8e1f', 'request_id': 'req-c2b29092-fb58-46ce-8378-996eab03722c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:08.149780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c2b29092-fb58-46ce-8378-996eab03722c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1d029424ba747d6a185dd0074df8e1f', 'request_id': 'req-c2b29092-fb58-46ce-8378-996eab03722c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:08.174960 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c2b29092-fb58-46ce-8378-996eab03722c admin admin] The token cache is being invalidate because project b1d029424ba747d6a185dd0074df8e1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:35:08.176388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c2b29092-fb58-46ce-8378-996eab03722c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:08.179633 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c2b29092-fb58-46ce-8378-996eab03722c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:08.186695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1238/2428] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:35:07 2022] DELETE /identity/v3/projects/b1d029424ba747d6a185dd0074df8e1f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:35:17.489250 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c9866-a5c6-450e-9762-54e2e25cb0de None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:17.490431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c9866-a5c6-450e-9762-54e2e25cb0de None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:17.490883 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c9866-a5c6-450e-9762-54e2e25cb0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:17.526750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-423c9866-a5c6-450e-9762-54e2e25cb0de None None] MFA Rules not processed for user `61709ccc1e244185b252179451cd860f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:17.528756 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-423c9866-a5c6-450e-9762-54e2e25cb0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:17.546476 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:17.643395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1191/2429] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 29 20:35:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:17.665520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b29092-fb58-46ce-8378-996eab03722c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:17.668541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c2b29092-fb58-46ce-8378-996eab03722c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:17.841140 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:17.934937 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b29092-fb58-46ce-8378-996eab03722c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:17.943224 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3d434-d86a-493c-9280-e6efe43216cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599129b7d8ff47efb145787414ddb80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:17.944375 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3d434-d86a-493c-9280-e6efe43216cf service swift] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:17.944375 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3d434-d86a-493c-9280-e6efe43216cf service swift] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:17.944375 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3d434-d86a-493c-9280-e6efe43216cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:17.975970 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3d434-d86a-493c-9280-e6efe43216cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:17.986792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3d434-d86a-493c-9280-e6efe43216cf service swift] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:18.006981 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1239/2430] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 29 20:35:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:35:18.421442 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc71a2c-d2df-4406-96db-aab5a37265c4 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:18.421442 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc71a2c-d2df-4406-96db-aab5a37265c4 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:18.421442 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc71a2c-d2df-4406-96db-aab5a37265c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:18.454638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ecc71a2c-d2df-4406-96db-aab5a37265c4 None None] MFA Rules not processed for user `8fbc5bd64315421e97931402fb0e3a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:18.459837 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ecc71a2c-d2df-4406-96db-aab5a37265c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:18.635123 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1192/2431] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:35:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:19.248845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38727d96-dd38-442c-97ac-4d84b3f416b9 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:19.249326 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38727d96-dd38-442c-97ac-4d84b3f416b9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:19.249418 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38727d96-dd38-442c-97ac-4d84b3f416b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:19.337795 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-38727d96-dd38-442c-97ac-4d84b3f416b9 None None] MFA Rules not processed for user `61709ccc1e244185b252179451cd860f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:19.340366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-38727d96-dd38-442c-97ac-4d84b3f416b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:19.371083 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1240/2432] 10.222.0.18 () {58 vars in 967 bytes} [Mon Aug 29 20:35:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:19.413511 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc71a2c-d2df-4406-96db-aab5a37265c4 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:19.434273 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc71a2c-d2df-4406-96db-aab5a37265c4 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:19.455853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70de6c-2311-482b-9f6a-5dba740a377f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599129b7d8ff47efb145787414ddb80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:19.456732 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70de6c-2311-482b-9f6a-5dba740a377f service swift] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:19.457180 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70de6c-2311-482b-9f6a-5dba740a377f service swift] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:19.457614 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70de6c-2311-482b-9f6a-5dba740a377f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:19.483410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70de6c-2311-482b-9f6a-5dba740a377f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:19.486006 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70de6c-2311-482b-9f6a-5dba740a377f service swift] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:19.518684 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1193/2433] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 29 20:35:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:35:20.231406 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051072f6-d3ae-4cb6-87f0-5d0d507b5f19 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:20.232193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051072f6-d3ae-4cb6-87f0-5d0d507b5f19 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:20.232998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051072f6-d3ae-4cb6-87f0-5d0d507b5f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:20.269171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-051072f6-d3ae-4cb6-87f0-5d0d507b5f19 None None] MFA Rules not processed for user `61709ccc1e244185b252179451cd860f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:20.272148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-051072f6-d3ae-4cb6-87f0-5d0d507b5f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:20.312237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1241/2434] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 29 20:35:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:20.343165 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70de6c-2311-482b-9f6a-5dba740a377f service swift] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:20.362598 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70de6c-2311-482b-9f6a-5dba740a377f service swift] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:20.374611 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095830d-f129-4aa4-bbcf-87248743f35a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599129b7d8ff47efb145787414ddb80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:20.375237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095830d-f129-4aa4-bbcf-87248743f35a service swift] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:20.375519 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095830d-f129-4aa4-bbcf-87248743f35a service swift] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:20.375794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095830d-f129-4aa4-bbcf-87248743f35a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:20.409599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095830d-f129-4aa4-bbcf-87248743f35a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:20.411767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095830d-f129-4aa4-bbcf-87248743f35a service swift] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:20.441252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1194/2435] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 29 20:35:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:35:20.565812 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3e655-3fdc-4c50-a1c9-8c7b461a5c13 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:20.566100 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3e655-3fdc-4c50-a1c9-8c7b461a5c13 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:20.566569 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3e655-3fdc-4c50-a1c9-8c7b461a5c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:20.640525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2fb3e655-3fdc-4c50-a1c9-8c7b461a5c13 None None] MFA Rules not processed for user `7f0b7046422a44e2acb959ad4878f079`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:20.643493 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2fb3e655-3fdc-4c50-a1c9-8c7b461a5c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:20.803481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1242/2436] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:35:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:21.207891 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1841e5-310a-4622-a4bc-dc34f23668f5 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:21.208774 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1841e5-310a-4622-a4bc-dc34f23668f5 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:21.209301 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1841e5-310a-4622-a4bc-dc34f23668f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:21.281456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-bb1841e5-310a-4622-a4bc-dc34f23668f5 None None] MFA Rules not processed for user `61709ccc1e244185b252179451cd860f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:21.287596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bb1841e5-310a-4622-a4bc-dc34f23668f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:21.325804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1195/2437] 10.222.0.18 () {58 vars in 967 bytes} [Mon Aug 29 20:35:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:21.346056 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3e655-3fdc-4c50-a1c9-8c7b461a5c13 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:21.382822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3e655-3fdc-4c50-a1c9-8c7b461a5c13 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:21.403629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21043933-524c-465f-8833-666a7ace2c3b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599129b7d8ff47efb145787414ddb80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:21.404547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21043933-524c-465f-8833-666a7ace2c3b service swift] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:21.405029 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21043933-524c-465f-8833-666a7ace2c3b service swift] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:21.405512 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21043933-524c-465f-8833-666a7ace2c3b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:21.418682 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21043933-524c-465f-8833-666a7ace2c3b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:21.421830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21043933-524c-465f-8833-666a7ace2c3b service swift] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:21.454223 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1243/2438] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 29 20:35:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 20:35:21.625958 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1841e5-310a-4622-a4bc-dc34f23668f5 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:21.627833 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bb1841e5-310a-4622-a4bc-dc34f23668f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:21.911223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1841e5-310a-4622-a4bc-dc34f23668f5 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:21.940816 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e55462-56b2-4381-832e-b4bbb3083eb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:21.941612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e55462-56b2-4381-832e-b4bbb3083eb3 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:21.941965 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e55462-56b2-4381-832e-b4bbb3083eb3 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:21.942307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e55462-56b2-4381-832e-b4bbb3083eb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:21.945004 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-91e55462-56b2-4381-832e-b4bbb3083eb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:22.067081 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:22.174937 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e55462-56b2-4381-832e-b4bbb3083eb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:22.176683 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e55462-56b2-4381-832e-b4bbb3083eb3 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:22.206115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1196/2439] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:35:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:35:25.573533 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21043933-524c-465f-8833-666a7ace2c3b service swift] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:25.575830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-21043933-524c-465f-8833-666a7ace2c3b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:25.705277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21043933-524c-465f-8833-666a7ace2c3b service swift] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:25.716914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae137f1a-f367-49f1-900e-5906d4ad4761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:25.717850 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae137f1a-f367-49f1-900e-5906d4ad4761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:25.718391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae137f1a-f367-49f1-900e-5906d4ad4761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:25.718921 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae137f1a-f367-49f1-900e-5906d4ad4761 admin admin] PATH_INFO: `/v3/users/39abe902a6534e80affc78acafc0ab31` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:25.721751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae137f1a-f367-49f1-900e-5906d4ad4761 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39abe902a6534e80affc78acafc0ab31)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:25.724162 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae137f1a-f367-49f1-900e-5906d4ad4761 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:25.767899 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:25.863849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ae137f1a-f367-49f1-900e-5906d4ad4761 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39abe902a6534e80affc78acafc0ab31'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:25.871394 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fb4e3-5b2c-4974-9ace-ef185c82745b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:25.871904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fb4e3-5b2c-4974-9ace-ef185c82745b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:25.872047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fb4e3-5b2c-4974-9ace-ef185c82745b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:25.887139 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ae137f1a-f367-49f1-900e-5906d4ad4761 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39abe902a6534e80affc78acafc0ab31'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:25.895884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-aa3fb4e3-5b2c-4974-9ace-ef185c82745b None None] MFA Rules not processed for user `b93ff419d60043f0aab0c11a6abdaa38`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:25.897952 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ae137f1a-f367-49f1-900e-5906d4ad4761 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39abe902a6534e80affc78acafc0ab31'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:25.898907 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ae137f1a-f367-49f1-900e-5906d4ad4761 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39abe902a6534e80affc78acafc0ab31'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:25.899077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-aa3fb4e3-5b2c-4974-9ace-ef185c82745b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:25.899967 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ae137f1a-f367-49f1-900e-5906d4ad4761 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39abe902a6534e80affc78acafc0ab31'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:25.907987 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1197/2440] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:35:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:25.925310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1244/2441] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:35:25 2022] DELETE /identity/v3/users/39abe902a6534e80affc78acafc0ab31 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:25.930339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536530a-2450-4558-978a-737090b0ff74 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:25.930625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536530a-2450-4558-978a-737090b0ff74 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:25.931638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536530a-2450-4558-978a-737090b0ff74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:25.958592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6536530a-2450-4558-978a-737090b0ff74 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:25.960368 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6536530a-2450-4558-978a-737090b0ff74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:26.081748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1198/2442] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:26.092686 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd53c1-bf49-40ae-acc3-2730d71b1f2a None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:26.092975 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd53c1-bf49-40ae-acc3-2730d71b1f2a None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:26.093254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd53c1-bf49-40ae-acc3-2730d71b1f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:26.101166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536530a-2450-4558-978a-737090b0ff74 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:26.104622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6536530a-2450-4558-978a-737090b0ff74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:26.122449 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-78dd53c1-bf49-40ae-acc3-2730d71b1f2a None None] MFA Rules not processed for user `448dc80e8c1b486fbdf83317209528eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:26.123670 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-78dd53c1-bf49-40ae-acc3-2730d71b1f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:26.217264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536530a-2450-4558-978a-737090b0ff74 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:26.221448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a131f55-7816-40a8-852b-8025eba12fa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:26.222079 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a131f55-7816-40a8-852b-8025eba12fa4 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:26.222355 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a131f55-7816-40a8-852b-8025eba12fa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:26.222854 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a131f55-7816-40a8-852b-8025eba12fa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:26.234039 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1245/2443] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:35:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:26.249315 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a131f55-7816-40a8-852b-8025eba12fa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:26.251152 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a131f55-7816-40a8-852b-8025eba12fa4 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:26.287028 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1199/2444] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:35:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:35:26.740314 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd53c1-bf49-40ae-acc3-2730d71b1f2a None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:26.743605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-78dd53c1-bf49-40ae-acc3-2730d71b1f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:26.772680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd53c1-bf49-40ae-acc3-2730d71b1f2a None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:26.787657 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12fda5-8856-4df3-a2db-68c5f6017668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:26.788530 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12fda5-8856-4df3-a2db-68c5f6017668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:26.788818 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12fda5-8856-4df3-a2db-68c5f6017668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:26.789097 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12fda5-8856-4df3-a2db-68c5f6017668 admin admin] PATH_INFO: `/v3/projects/e4de197062eb48adb3f8af1a1f8e958c` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:26.793139 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12fda5-8856-4df3-a2db-68c5f6017668 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4de197062eb48adb3f8af1a1f8e958c)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:26.796440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12fda5-8856-4df3-a2db-68c5f6017668 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:26.877469 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-2d12fda5-8856-4df3-a2db-68c5f6017668 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4de197062eb48adb3f8af1a1f8e958c', 'request_id': 'req-2d12fda5-8856-4df3-a2db-68c5f6017668'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:26.893421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-2d12fda5-8856-4df3-a2db-68c5f6017668 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4de197062eb48adb3f8af1a1f8e958c', 'request_id': 'req-2d12fda5-8856-4df3-a2db-68c5f6017668'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:26.904642 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-2d12fda5-8856-4df3-a2db-68c5f6017668 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4de197062eb48adb3f8af1a1f8e958c', 'request_id': 'req-2d12fda5-8856-4df3-a2db-68c5f6017668'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:26.938159 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-2d12fda5-8856-4df3-a2db-68c5f6017668 admin admin] The token cache is being invalidate because project e4de197062eb48adb3f8af1a1f8e958c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:35:26.938955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-2d12fda5-8856-4df3-a2db-68c5f6017668 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:26.940417 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-2d12fda5-8856-4df3-a2db-68c5f6017668 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:35:26.946398 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1246/2445] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:35:26 2022] DELETE /identity/v3/projects/e4de197062eb48adb3f8af1a1f8e958c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:26.948960 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9035429-857d-4c2f-b52c-1d6708e44af5 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:26.950184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9035429-857d-4c2f-b52c-1d6708e44af5 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:26.950660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9035429-857d-4c2f-b52c-1d6708e44af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:26.977906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d9035429-857d-4c2f-b52c-1d6708e44af5 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:26.980783 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d9035429-857d-4c2f-b52c-1d6708e44af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:27.121003 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1200/2446] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:35:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:27.129692 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd48ec2-f4a7-41f5-b861-57b1d4e4b629 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:27.130017 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd48ec2-f4a7-41f5-b861-57b1d4e4b629 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:27.130527 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd48ec2-f4a7-41f5-b861-57b1d4e4b629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:27.163257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-6dd48ec2-f4a7-41f5-b861-57b1d4e4b629 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:27.164648 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6dd48ec2-f4a7-41f5-b861-57b1d4e4b629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:27.177136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1247/2447] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:27.182705 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9035429-857d-4c2f-b52c-1d6708e44af5 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:27.195274 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9035429-857d-4c2f-b52c-1d6708e44af5 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:27.203086 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8e9d9-d398-4818-a0ff-f8131d81ad18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:27.204036 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8e9d9-d398-4818-a0ff-f8131d81ad18 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:27.204484 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8e9d9-d398-4818-a0ff-f8131d81ad18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:27.204892 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8e9d9-d398-4818-a0ff-f8131d81ad18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:27.205617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f8e9d9-d398-4818-a0ff-f8131d81ad18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:35:27.205926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f8e9d9-d398-4818-a0ff-f8131d81ad18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:27.207664 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f8e9d9-d398-4818-a0ff-f8131d81ad18 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:27.209819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:27.235112 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1201/2448] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:35:27.236219 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd48ec2-f4a7-41f5-b861-57b1d4e4b629 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:27.248942 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd48ec2-f4a7-41f5-b861-57b1d4e4b629 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:27.257904 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194988c-fba8-47be-bf26-c027605e12ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:27.258922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194988c-fba8-47be-bf26-c027605e12ed admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:27.259433 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194988c-fba8-47be-bf26-c027605e12ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:27.259963 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194988c-fba8-47be-bf26-c027605e12ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:27.261447 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194988c-fba8-47be-bf26-c027605e12ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:27.264372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194988c-fba8-47be-bf26-c027605e12ed admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:27.293991 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:27.321403 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8e9d9-d398-4818-a0ff-f8131d81ad18 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:27.327652 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1248/2449] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:35:27 2022] POST /identity/v3/projects => generated 377 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:35:27.337237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8e9d9-d398-4818-a0ff-f8131d81ad18 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:27.418763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024c4ee-6f7e-457c-95ef-a15ddb2ad03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:27.420043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024c4ee-6f7e-457c-95ef-a15ddb2ad03f admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:27.420780 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024c4ee-6f7e-457c-95ef-a15ddb2ad03f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:27.421357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024c4ee-6f7e-457c-95ef-a15ddb2ad03f admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:27.422191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024c4ee-6f7e-457c-95ef-a15ddb2ad03f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:27.427136 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024c4ee-6f7e-457c-95ef-a15ddb2ad03f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:27.455307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1202/2450] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:35:27 2022] POST /identity/v3/users => generated 289 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:35:27.460112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194988c-fba8-47be-bf26-c027605e12ed admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:27.478034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194988c-fba8-47be-bf26-c027605e12ed admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:27.485992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958383c5-2442-4332-8e59-ddb01f428558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:27.486413 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958383c5-2442-4332-8e59-ddb01f428558 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:27.486700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958383c5-2442-4332-8e59-ddb01f428558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:27.486982 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958383c5-2442-4332-8e59-ddb01f428558 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:27.487638 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958383c5-2442-4332-8e59-ddb01f428558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:27.489643 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958383c5-2442-4332-8e59-ddb01f428558 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:27.515980 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1249/2451] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:35:27.519752 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024c4ee-6f7e-457c-95ef-a15ddb2ad03f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:27.531881 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024c4ee-6f7e-457c-95ef-a15ddb2ad03f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:27.539714 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1147336-b7df-4930-b67a-bacbe0dc44cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:27.540403 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1147336-b7df-4930-b67a-bacbe0dc44cf admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:27.540681 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1147336-b7df-4930-b67a-bacbe0dc44cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:27.541104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1147336-b7df-4930-b67a-bacbe0dc44cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:27.544466 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1147336-b7df-4930-b67a-bacbe0dc44cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:27.546677 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1147336-b7df-4930-b67a-bacbe0dc44cf admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:27.564152 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1203/2452] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:35:27.565321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958383c5-2442-4332-8e59-ddb01f428558 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:27.602624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958383c5-2442-4332-8e59-ddb01f428558 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:27.608904 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59838403-5534-4c7b-9301-7985731a014c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:27.609805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59838403-5534-4c7b-9301-7985731a014c admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:27.610134 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59838403-5534-4c7b-9301-7985731a014c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:27.610477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59838403-5534-4c7b-9301-7985731a014c admin admin] PATH_INFO: `/v3/projects/4ac573b4aa7a47c49d129afd9939e26d/users/fca06f41184444ccb0ddb57e5d44f227/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:27.630360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59838403-5534-4c7b-9301-7985731a014c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ac573b4aa7a47c49d129afd9939e26d, user_id=fca06f41184444ccb0ddb57e5d44f227, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:27.633310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59838403-5534-4c7b-9301-7985731a014c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:27.659985 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1250/2453] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:35:27 2022] PUT /identity/v3/projects/4ac573b4aa7a47c49d129afd9939e26d/users/fca06f41184444ccb0ddb57e5d44f227/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:27.666272 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7503b-2790-4c84-bb1b-329967a1b598 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:27.667389 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7503b-2790-4c84-bb1b-329967a1b598 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:27.667668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7503b-2790-4c84-bb1b-329967a1b598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:27.692471 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-cbc7503b-2790-4c84-bb1b-329967a1b598 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:27.694108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-cbc7503b-2790-4c84-bb1b-329967a1b598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:27.789045 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1204/2454] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:27.816683 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59838403-5534-4c7b-9301-7985731a014c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:27.819116 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-59838403-5534-4c7b-9301-7985731a014c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:27.903640 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59838403-5534-4c7b-9301-7985731a014c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:27.912775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309531b-a393-4a92-bd87-b060d88a48b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:27.913496 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309531b-a393-4a92-bd87-b060d88a48b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:27.913857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309531b-a393-4a92-bd87-b060d88a48b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:27.914515 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f309531b-a393-4a92-bd87-b060d88a48b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:27.925495 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309531b-a393-4a92-bd87-b060d88a48b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:27.927046 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f309531b-a393-4a92-bd87-b060d88a48b7 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:27.942125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1251/2455] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:35:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:35:33.758933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d6ea5-02d7-46d7-928a-dc665053b907 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:33.760735 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d6ea5-02d7-46d7-928a-dc665053b907 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:33.761330 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d6ea5-02d7-46d7-928a-dc665053b907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:33.801819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b72d6ea5-02d7-46d7-928a-dc665053b907 None None] MFA Rules not processed for user `61709ccc1e244185b252179451cd860f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:33.814289 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b72d6ea5-02d7-46d7-928a-dc665053b907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:33.928398 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1205/2456] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 29 20:35:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:33.942840 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309531b-a393-4a92-bd87-b060d88a48b7 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:33.945194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f309531b-a393-4a92-bd87-b060d88a48b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:34.074251 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f309531b-a393-4a92-bd87-b060d88a48b7 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:34.080387 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc0d73b-bd6d-4436-a85b-03ab6884c458 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599129b7d8ff47efb145787414ddb80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:34.084624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0d73b-bd6d-4436-a85b-03ab6884c458 service swift] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:34.085243 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0d73b-bd6d-4436-a85b-03ab6884c458 service swift] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:34.085660 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0d73b-bd6d-4436-a85b-03ab6884c458 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:34.098775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc0d73b-bd6d-4436-a85b-03ab6884c458 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:34.103858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc0d73b-bd6d-4436-a85b-03ab6884c458 service swift] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:34.117763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1252/2457] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 29 20:35:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:35:36.610692 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96e3be-6f9e-4a0a-b586-75370245133c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:36.612807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96e3be-6f9e-4a0a-b586-75370245133c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:36.613334 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96e3be-6f9e-4a0a-b586-75370245133c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:36.659799 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-bf96e3be-6f9e-4a0a-b586-75370245133c None None] MFA Rules not processed for user `fca06f41184444ccb0ddb57e5d44f227`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:36.661674 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bf96e3be-6f9e-4a0a-b586-75370245133c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:36.669744 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:36.791160 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1206/2458] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:35:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:35:36.796102 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc0d73b-bd6d-4436-a85b-03ab6884c458 service swift] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:36.809139 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc0d73b-bd6d-4436-a85b-03ab6884c458 service swift] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:36.819504 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686567d-48d5-4181-af84-e31fda76aa6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:36.820262 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686567d-48d5-4181-af84-e31fda76aa6b admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:36.820645 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686567d-48d5-4181-af84-e31fda76aa6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:36.821018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686567d-48d5-4181-af84-e31fda76aa6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:36.821760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686567d-48d5-4181-af84-e31fda76aa6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:36.824476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686567d-48d5-4181-af84-e31fda76aa6b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:36.924259 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1253/2459] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:35:36 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:35:36.925806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96e3be-6f9e-4a0a-b586-75370245133c None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:36.946478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96e3be-6f9e-4a0a-b586-75370245133c None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:37.097373 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4db465-c791-47f7-8ba6-f31c9c8d450e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:37.098136 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4db465-c791-47f7-8ba6-f31c9c8d450e admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:37.101860 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4db465-c791-47f7-8ba6-f31c9c8d450e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:37.102261 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4db465-c791-47f7-8ba6-f31c9c8d450e admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:37.102935 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4db465-c791-47f7-8ba6-f31c9c8d450e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:37.105412 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4db465-c791-47f7-8ba6-f31c9c8d450e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:37.149860 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1207/2460] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:35:36 2022] POST /identity/v3/users => generated 295 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:35:37.154510 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686567d-48d5-4181-af84-e31fda76aa6b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:37.167692 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686567d-48d5-4181-af84-e31fda76aa6b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:37.173315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de3593-afc0-4d51-af90-999019c9575e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:37.173961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de3593-afc0-4d51-af90-999019c9575e admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:37.174287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de3593-afc0-4d51-af90-999019c9575e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:37.174677 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de3593-afc0-4d51-af90-999019c9575e admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:37.175685 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de3593-afc0-4d51-af90-999019c9575e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:37.187490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de3593-afc0-4d51-af90-999019c9575e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:37.204735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1254/2461] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:35:37.205943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4db465-c791-47f7-8ba6-f31c9c8d450e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:37.225740 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4db465-c791-47f7-8ba6-f31c9c8d450e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:37.237523 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f65a3d-907b-484d-9caf-979c7f7d609f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:37.238557 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f65a3d-907b-484d-9caf-979c7f7d609f admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:37.238996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f65a3d-907b-484d-9caf-979c7f7d609f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:37.239450 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f65a3d-907b-484d-9caf-979c7f7d609f admin admin] PATH_INFO: `/v3/projects/f885e578ed46463dad421c9f1ecc6520/users/9a07538a219c4182b0a21d157a90c60d/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:37.259287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f65a3d-907b-484d-9caf-979c7f7d609f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f885e578ed46463dad421c9f1ecc6520, user_id=9a07538a219c4182b0a21d157a90c60d, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:37.262579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f65a3d-907b-484d-9caf-979c7f7d609f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:37.282695 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1208/2462] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:35:37 2022] PUT /identity/v3/projects/f885e578ed46463dad421c9f1ecc6520/users/9a07538a219c4182b0a21d157a90c60d/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:35:37.292086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de3593-afc0-4d51-af90-999019c9575e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:37.310593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de3593-afc0-4d51-af90-999019c9575e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:37.329094 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:37.415032 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2ee1f-288f-4254-b244-c46c40b81002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:37.415954 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2ee1f-288f-4254-b244-c46c40b81002 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:37.417321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2ee1f-288f-4254-b244-c46c40b81002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:37.417743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2ee1f-288f-4254-b244-c46c40b81002 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:37.418557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2ee1f-288f-4254-b244-c46c40b81002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:37.420385 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2ee1f-288f-4254-b244-c46c40b81002 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:37.448309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1255/2463] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:35:37.456000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f65a3d-907b-484d-9caf-979c7f7d609f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:37.468514 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f65a3d-907b-484d-9caf-979c7f7d609f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:37.476592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8b874-e196-4d8e-800f-d61964e27d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:37.477599 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8b874-e196-4d8e-800f-d61964e27d3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:37.477979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8b874-e196-4d8e-800f-d61964e27d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:37.478371 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8b874-e196-4d8e-800f-d61964e27d3f admin admin] PATH_INFO: `/v3/projects/f885e578ed46463dad421c9f1ecc6520/users/9a07538a219c4182b0a21d157a90c60d/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:37.481082 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa8b874-e196-4d8e-800f-d61964e27d3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f885e578ed46463dad421c9f1ecc6520, user_id=9a07538a219c4182b0a21d157a90c60d, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:37.484151 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa8b874-e196-4d8e-800f-d61964e27d3f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:37.539824 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1209/2464] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:35:37 2022] PUT /identity/v3/projects/f885e578ed46463dad421c9f1ecc6520/users/9a07538a219c4182b0a21d157a90c60d/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:35:46.402115 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc91d7-1152-4e70-835b-594427594e80 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:46.402788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc91d7-1152-4e70-835b-594427594e80 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:46.402788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc91d7-1152-4e70-835b-594427594e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:46.465161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-34fc91d7-1152-4e70-835b-594427594e80 None None] MFA Rules not processed for user `9a07538a219c4182b0a21d157a90c60d`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:46.466830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-34fc91d7-1152-4e70-835b-594427594e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:46.562120 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1256/2465] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:35:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:46.576872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8b874-e196-4d8e-800f-d61964e27d3f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:46.579067 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-efa8b874-e196-4d8e-800f-d61964e27d3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:46.678372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:46.705739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8b874-e196-4d8e-800f-d61964e27d3f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:46.714333 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca441309-5a7d-427b-97ac-bb2c12d5d881 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:46.714958 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca441309-5a7d-427b-97ac-bb2c12d5d881 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:46.715238 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca441309-5a7d-427b-97ac-bb2c12d5d881 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:46.715509 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca441309-5a7d-427b-97ac-bb2c12d5d881 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:46.820803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca441309-5a7d-427b-97ac-bb2c12d5d881 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:46.823989 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca441309-5a7d-427b-97ac-bb2c12d5d881 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:46.832924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1210/2466] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:35:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:35:53.104445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d16c8-1bb9-4278-8c10-ac31ffe27675 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:53.104445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d16c8-1bb9-4278-8c10-ac31ffe27675 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:53.104445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d16c8-1bb9-4278-8c10-ac31ffe27675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:53.125649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-553d16c8-1bb9-4278-8c10-ac31ffe27675 None None] MFA Rules not processed for user `93dd431edc40415295518d9d24fd94af`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:53.127416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-553d16c8-1bb9-4278-8c10-ac31ffe27675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:53.228771 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1257/2467] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:35:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:54.175974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb135f-c33d-4ff1-987e-b278aa265ca2 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:54.176887 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb135f-c33d-4ff1-987e-b278aa265ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:54.177321 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb135f-c33d-4ff1-987e-b278aa265ca2 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:54.179018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1211/2468] 10.222.0.18 () {54 vars in 961 bytes} [Mon Aug 29 20:35:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:35:54.185332 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a31a5-5bed-41dd-9290-f55c4abf2ba5 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:54.185895 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a31a5-5bed-41dd-9290-f55c4abf2ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:54.186016 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a31a5-5bed-41dd-9290-f55c4abf2ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:54.225108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d14a31a5-5bed-41dd-9290-f55c4abf2ba5 None None] MFA Rules not processed for user `fa0e659eee5e4a9da8d3d7337cb73798`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:35:54.226458 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d14a31a5-5bed-41dd-9290-f55c4abf2ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:54.304791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:35:54.337857 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1258/2469] 10.222.0.18 () {58 vars in 1058 bytes} [Mon Aug 29 20:35:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:35:54.349933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb135f-c33d-4ff1-987e-b278aa265ca2 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:35:54.368950 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb135f-c33d-4ff1-987e-b278aa265ca2 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:35:54.379960 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5a45d-eab8-4b99-9797-07aebc9f2519 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0e659eee5e4a9da8d3d7337cb73798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:35:54.380906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5a45d-eab8-4b99-9797-07aebc9f2519 service placement] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:35:54.381372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5a45d-eab8-4b99-9797-07aebc9f2519 service placement] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:35:54.381660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5a45d-eab8-4b99-9797-07aebc9f2519 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:35:54.383580 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9ac5a45d-eab8-4b99-9797-07aebc9f2519 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:35:54.398661 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5a45d-eab8-4b99-9797-07aebc9f2519 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:35:54.401992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5a45d-eab8-4b99-9797-07aebc9f2519 service placement] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:35:54.422038 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1212/2470] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:35:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:36:04.326632 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=9 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:04.414719 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=35 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:07.914415 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a31a5-5bed-41dd-9290-f55c4abf2ba5 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:07.915829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d14a31a5-5bed-41dd-9290-f55c4abf2ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:07.973946 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a31a5-5bed-41dd-9290-f55c4abf2ba5 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:07.977422 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d1c4d-b15d-404b-ae84-7a19ab55a9b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:07.978038 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d1c4d-b15d-404b-ae84-7a19ab55a9b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:07.978315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d1c4d-b15d-404b-ae84-7a19ab55a9b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:07.978588 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61d1c4d-b15d-404b-ae84-7a19ab55a9b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:07.981020 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c61d1c4d-b15d-404b-ae84-7a19ab55a9b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:08.055279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d1c4d-b15d-404b-ae84-7a19ab55a9b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:08.056789 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61d1c4d-b15d-404b-ae84-7a19ab55a9b7 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:08.069011 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1259/2471] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:36:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:36:11.169749 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0fa3c-fc3b-4542-9350-32ead1c90cd1 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:11.170225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0fa3c-fc3b-4542-9350-32ead1c90cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:11.170225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0fa3c-fc3b-4542-9350-32ead1c90cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:11.190346 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-26b0fa3c-fc3b-4542-9350-32ead1c90cd1 None None] MFA Rules not processed for user `fca06f41184444ccb0ddb57e5d44f227`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:36:11.191657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-26b0fa3c-fc3b-4542-9350-32ead1c90cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:11.199831 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1213/2472] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:36:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:36:13.864730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d1c4d-b15d-404b-ae84-7a19ab55a9b7 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:13.873427 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61d1c4d-b15d-404b-ae84-7a19ab55a9b7 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:13.877054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c309ac-e49d-4b75-9979-2410eccf727f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:13.877661 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c309ac-e49d-4b75-9979-2410eccf727f service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:13.877941 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c309ac-e49d-4b75-9979-2410eccf727f service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:13.878211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c309ac-e49d-4b75-9979-2410eccf727f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:13.879928 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-03c309ac-e49d-4b75-9979-2410eccf727f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:13.960808 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c309ac-e49d-4b75-9979-2410eccf727f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:13.962340 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c309ac-e49d-4b75-9979-2410eccf727f service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:13.975906 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1260/2473] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:36:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:36:20.620732 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396321d-9079-4a60-a1b4-d6a2a9030261 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:20.621408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396321d-9079-4a60-a1b4-d6a2a9030261 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:20.621408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396321d-9079-4a60-a1b4-d6a2a9030261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:20.640912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-2396321d-9079-4a60-a1b4-d6a2a9030261 None None] MFA Rules not processed for user `61709ccc1e244185b252179451cd860f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:36:20.642213 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2396321d-9079-4a60-a1b4-d6a2a9030261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:20.702309 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1214/2474] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 29 20:36:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:36:20.724936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c309ac-e49d-4b75-9979-2410eccf727f service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:20.774648 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:20.797317 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c309ac-e49d-4b75-9979-2410eccf727f service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:20.803034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c1b09-218d-4e62-b525-89877ebfe87a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599129b7d8ff47efb145787414ddb80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:20.803886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c1b09-218d-4e62-b525-89877ebfe87a service swift] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:20.804339 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c1b09-218d-4e62-b525-89877ebfe87a service swift] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:20.804819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c1b09-218d-4e62-b525-89877ebfe87a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:20.815066 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c1b09-218d-4e62-b525-89877ebfe87a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:20.816739 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c1b09-218d-4e62-b525-89877ebfe87a service swift] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:20.825972 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1261/2475] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 29 20:36:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:36:30.693478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=28 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:30.822267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=12 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:34.117523 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396321d-9079-4a60-a1b4-d6a2a9030261 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:34.129163 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396321d-9079-4a60-a1b4-d6a2a9030261 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:34.133311 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3d2fb-8566-4d7a-b8c0-634f47f57977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:34.133989 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3d2fb-8566-4d7a-b8c0-634f47f57977 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:34.134316 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3d2fb-8566-4d7a-b8c0-634f47f57977 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:34.134630 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3d2fb-8566-4d7a-b8c0-634f47f57977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:34.146828 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3d2fb-8566-4d7a-b8c0-634f47f57977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:34.148890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3d2fb-8566-4d7a-b8c0-634f47f57977 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:34.161511 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1215/2476] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:36:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:36:40.055486 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a54c87-6480-44e5-af73-c373e060961d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:40.056099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a54c87-6480-44e5-af73-c373e060961d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:40.056099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a54c87-6480-44e5-af73-c373e060961d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:40.078648 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-87a54c87-6480-44e5-af73-c373e060961d None None] MFA Rules not processed for user `61709ccc1e244185b252179451cd860f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:36:40.079867 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-87a54c87-6480-44e5-af73-c373e060961d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:40.088980 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1262/2477] 10.222.0.18 () {58 vars in 967 bytes} [Mon Aug 29 20:36:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:36:40.110494 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3d2fb-8566-4d7a-b8c0-634f47f57977 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:40.126689 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3d2fb-8566-4d7a-b8c0-634f47f57977 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:40.137000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f6914-892f-4627-a1eb-5577c9b0092b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599129b7d8ff47efb145787414ddb80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:40.138084 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f6914-892f-4627-a1eb-5577c9b0092b service swift] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:40.138499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f6914-892f-4627-a1eb-5577c9b0092b service swift] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:40.138994 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f6914-892f-4627-a1eb-5577c9b0092b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:40.154702 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f6914-892f-4627-a1eb-5577c9b0092b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:40.157798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f6914-892f-4627-a1eb-5577c9b0092b service swift] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:40.168565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1216/2478] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 29 20:36:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 29 20:36:42.294421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a54c87-6480-44e5-af73-c373e060961d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:42.296047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-87a54c87-6480-44e5-af73-c373e060961d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:42.384717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a54c87-6480-44e5-af73-c373e060961d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:42.389385 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063112d-c4b6-481e-a9b6-a420def1be1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:42.390024 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063112d-c4b6-481e-a9b6-a420def1be1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:42.390308 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063112d-c4b6-481e-a9b6-a420def1be1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:42.390580 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063112d-c4b6-481e-a9b6-a420def1be1e admin admin] PATH_INFO: `/v3/users/93dd431edc40415295518d9d24fd94af` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:42.391717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063112d-c4b6-481e-a9b6-a420def1be1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=93dd431edc40415295518d9d24fd94af)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:42.393663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063112d-c4b6-481e-a9b6-a420def1be1e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:42.444632 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6063112d-c4b6-481e-a9b6-a420def1be1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93dd431edc40415295518d9d24fd94af'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:42.452139 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6063112d-c4b6-481e-a9b6-a420def1be1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93dd431edc40415295518d9d24fd94af'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:42.460422 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6063112d-c4b6-481e-a9b6-a420def1be1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93dd431edc40415295518d9d24fd94af'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:42.461072 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6063112d-c4b6-481e-a9b6-a420def1be1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93dd431edc40415295518d9d24fd94af'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:42.461749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6063112d-c4b6-481e-a9b6-a420def1be1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93dd431edc40415295518d9d24fd94af'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:42.480406 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1263/2479] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:36:42 2022] DELETE /identity/v3/users/93dd431edc40415295518d9d24fd94af => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:36:42.706790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f6914-892f-4627-a1eb-5577c9b0092b service swift] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:42.734544 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f6914-892f-4627-a1eb-5577c9b0092b service swift] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:42.821550 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c9781-1be1-46e4-9559-19eb987a67c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:42.822395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c9781-1be1-46e4-9559-19eb987a67c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:42.823218 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c9781-1be1-46e4-9559-19eb987a67c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:42.823469 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c9781-1be1-46e4-9559-19eb987a67c3 admin admin] PATH_INFO: `/v3/projects/1c85b91350b74daeab5ba89364b4578e` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:42.824626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c9781-1be1-46e4-9559-19eb987a67c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c85b91350b74daeab5ba89364b4578e)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:42.827067 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c9781-1be1-46e4-9559-19eb987a67c3 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:42.895687 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-416c9781-1be1-46e4-9559-19eb987a67c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c85b91350b74daeab5ba89364b4578e', 'request_id': 'req-416c9781-1be1-46e4-9559-19eb987a67c3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:42.902014 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-416c9781-1be1-46e4-9559-19eb987a67c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c85b91350b74daeab5ba89364b4578e', 'request_id': 'req-416c9781-1be1-46e4-9559-19eb987a67c3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:42.911841 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-416c9781-1be1-46e4-9559-19eb987a67c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c85b91350b74daeab5ba89364b4578e', 'request_id': 'req-416c9781-1be1-46e4-9559-19eb987a67c3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:42.924100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-416c9781-1be1-46e4-9559-19eb987a67c3 admin admin] The token cache is being invalidate because project 1c85b91350b74daeab5ba89364b4578e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:36:42.924381 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-416c9781-1be1-46e4-9559-19eb987a67c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:42.925040 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-416c9781-1be1-46e4-9559-19eb987a67c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:36:42.926550 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1217/2480] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:36:42 2022] DELETE /identity/v3/projects/1c85b91350b74daeab5ba89364b4578e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:36:46.244194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6eba0-4419-44ea-8a3b-c20db14826dc None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:46.244657 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6eba0-4419-44ea-8a3b-c20db14826dc None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:46.244801 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6eba0-4419-44ea-8a3b-c20db14826dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:46.264389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-93d6eba0-4419-44ea-8a3b-c20db14826dc None None] MFA Rules not processed for user `61709ccc1e244185b252179451cd860f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:36:46.265610 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-93d6eba0-4419-44ea-8a3b-c20db14826dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:46.325976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1264/2481] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 29 20:36:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:36:46.339023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c9781-1be1-46e4-9559-19eb987a67c3 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:36:46.340506 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-416c9781-1be1-46e4-9559-19eb987a67c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:36:46.411291 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c9781-1be1-46e4-9559-19eb987a67c3 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:36:46.416976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b287fb9-cbf0-4f00-a909-807872c5cfc5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599129b7d8ff47efb145787414ddb80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:36:46.417621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b287fb9-cbf0-4f00-a909-807872c5cfc5 service swift] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:36:46.417912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b287fb9-cbf0-4f00-a909-807872c5cfc5 service swift] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:36:46.418181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b287fb9-cbf0-4f00-a909-807872c5cfc5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:36:46.423574 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:46.429600 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b287fb9-cbf0-4f00-a909-807872c5cfc5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:36:46.432121 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b287fb9-cbf0-4f00-a909-807872c5cfc5 service swift] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:36:46.441413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1218/2482] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 29 20:36:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:36:56.318380 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=81,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:36:56.438036 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:37:00.931554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8933de-0702-408b-9690-399080cf37ed None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:00.932117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8933de-0702-408b-9690-399080cf37ed None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:00.932215 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8933de-0702-408b-9690-399080cf37ed None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:00.934142 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1265/2483] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 29 20:37:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:37:00.940366 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc8183-34dc-4e52-bbc5-b237237bd5d1 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:00.940857 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc8183-34dc-4e52-bbc5-b237237bd5d1 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:00.940954 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc8183-34dc-4e52-bbc5-b237237bd5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:00.970920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-35fc8183-34dc-4e52-bbc5-b237237bd5d1 None None] MFA Rules not processed for user `c0c44754a1c04a2b87081aba6cb509d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:37:00.972212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-35fc8183-34dc-4e52-bbc5-b237237bd5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:01.043813 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1219/2484] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 29 20:37:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:37:01.056052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8933de-0702-408b-9690-399080cf37ed None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:01.061231 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6a8933de-0702-408b-9690-399080cf37ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:01.118584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8933de-0702-408b-9690-399080cf37ed None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:01.122884 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f22687-4d1c-411b-a057-b7ec63393368 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d022ed8d2ac43ae816aa396a8e2f17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:01.123504 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f22687-4d1c-411b-a057-b7ec63393368 service glance] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:01.123766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f22687-4d1c-411b-a057-b7ec63393368 service glance] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:01.124073 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f22687-4d1c-411b-a057-b7ec63393368 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:01.133995 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f22687-4d1c-411b-a057-b7ec63393368 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:01.135411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f22687-4d1c-411b-a057-b7ec63393368 service glance] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:01.146080 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1266/2485] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:37:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:37:01.214916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266b71b-36ca-42b1-8026-a97673fced5c None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:01.215200 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266b71b-36ca-42b1-8026-a97673fced5c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:01.215471 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c266b71b-36ca-42b1-8026-a97673fced5c None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:01.216895 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1220/2486] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 29 20:37:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:37:01.222359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec49b0-fba2-4df2-a7e1-34fc9834293e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:01.222700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec49b0-fba2-4df2-a7e1-34fc9834293e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:01.222982 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec49b0-fba2-4df2-a7e1-34fc9834293e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:01.243167 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-33ec49b0-fba2-4df2-a7e1-34fc9834293e None None] MFA Rules not processed for user `d51e331a4c9c4f038bdb80f8c5b74e42`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:37:01.244541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-33ec49b0-fba2-4df2-a7e1-34fc9834293e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:01.301208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1267/2487] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 29 20:37:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:37:01.315063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266b71b-36ca-42b1-8026-a97673fced5c None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:01.316853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c266b71b-36ca-42b1-8026-a97673fced5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:01.330809 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c266b71b-36ca-42b1-8026-a97673fced5c None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:01.336773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ef5f4-0e9c-4fec-8d4b-9d743334c618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:01.337376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ef5f4-0e9c-4fec-8d4b-9d743334c618 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:01.337656 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ef5f4-0e9c-4fec-8d4b-9d743334c618 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:01.337923 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ef5f4-0e9c-4fec-8d4b-9d743334c618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:01.347629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ef5f4-0e9c-4fec-8d4b-9d743334c618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:01.349171 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ef5f4-0e9c-4fec-8d4b-9d743334c618 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:01.359580 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1221/2488] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:37:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:37:01.460157 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96060bf1-24c6-4a6d-a1f0-e7ce28eab926 None None] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:01.460442 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96060bf1-24c6-4a6d-a1f0-e7ce28eab926 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:01.460714 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96060bf1-24c6-4a6d-a1f0-e7ce28eab926 None None] PATH_INFO: `/` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:01.462086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1268/2489] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 29 20:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:37:01.468330 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15077e9-2a62-44d2-9a07-21e33d92cff8 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:01.468641 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15077e9-2a62-44d2-9a07-21e33d92cff8 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:01.468912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15077e9-2a62-44d2-9a07-21e33d92cff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:01.490989 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c15077e9-2a62-44d2-9a07-21e33d92cff8 None None] MFA Rules not processed for user `c0c44754a1c04a2b87081aba6cb509d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:37:01.492327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c15077e9-2a62-44d2-9a07-21e33d92cff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:01.502346 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1222/2490] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 29 20:37:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:37:01.512895 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96060bf1-24c6-4a6d-a1f0-e7ce28eab926 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:01.523844 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96060bf1-24c6-4a6d-a1f0-e7ce28eab926 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:01.528091 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084d1b6-43e5-49af-867e-274cd682820d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:01.528703 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084d1b6-43e5-49af-867e-274cd682820d service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:01.528984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084d1b6-43e5-49af-867e-274cd682820d service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:01.529257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084d1b6-43e5-49af-867e-274cd682820d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:01.540608 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084d1b6-43e5-49af-867e-274cd682820d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:01.542644 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084d1b6-43e5-49af-867e-274cd682820d service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:01.555694 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1269/2491] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:37:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:37:04.635182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6fb78-655f-40fc-9885-790a5ce574c1 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:04.638499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6fb78-655f-40fc-9885-790a5ce574c1 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:04.638499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6fb78-655f-40fc-9885-790a5ce574c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:04.659988 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-99c6fb78-655f-40fc-9885-790a5ce574c1 None None] MFA Rules not processed for user `61709ccc1e244185b252179451cd860f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:37:04.661637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-99c6fb78-655f-40fc-9885-790a5ce574c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:04.669550 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1223/2492] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 29 20:37:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:37:04.682470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084d1b6-43e5-49af-867e-274cd682820d service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:04.697859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084d1b6-43e5-49af-867e-274cd682820d service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:04.703223 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd8451-7f7b-43e3-861c-8b107b5a49ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599129b7d8ff47efb145787414ddb80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:04.703822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd8451-7f7b-43e3-861c-8b107b5a49ea service swift] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:04.704131 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd8451-7f7b-43e3-861c-8b107b5a49ea service swift] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:04.704401 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd8451-7f7b-43e3-861c-8b107b5a49ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:04.712419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd8451-7f7b-43e3-861c-8b107b5a49ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:04.713982 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd8451-7f7b-43e3-861c-8b107b5a49ea service swift] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:04.723130 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1270/2493] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 29 20:37:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:37:14.654617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=43 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:37:14.718794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=61 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:37:27.510676 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b942d-87df-41d4-894c-c974d25a8547 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:27.511192 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b942d-87df-41d4-894c-c974d25a8547 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:27.511300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b942d-87df-41d4-894c-c974d25a8547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:27.532622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-053b942d-87df-41d4-894c-c974d25a8547 None None] MFA Rules not processed for user `61709ccc1e244185b252179451cd860f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:37:27.533857 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-053b942d-87df-41d4-894c-c974d25a8547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:27.542039 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1224/2494] 10.222.0.18 () {58 vars in 967 bytes} [Mon Aug 29 20:37:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:37:27.565236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd8451-7f7b-43e3-861c-8b107b5a49ea service swift] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:27.576249 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd8451-7f7b-43e3-861c-8b107b5a49ea service swift] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:27.581431 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e2e59-c076-454d-a826-674daf5fe9a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599129b7d8ff47efb145787414ddb80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:27.582052 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e2e59-c076-454d-a826-674daf5fe9a1 service swift] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:27.582335 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e2e59-c076-454d-a826-674daf5fe9a1 service swift] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:27.582609 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e2e59-c076-454d-a826-674daf5fe9a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:27.591682 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e2e59-c076-454d-a826-674daf5fe9a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:27.593278 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e2e59-c076-454d-a826-674daf5fe9a1 service swift] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:27.601620 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1271/2495] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 29 20:37:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:37:37.525784 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:37:37.598436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:37:42.229066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cc507-6e9a-43b1-aa32-dbd0666771c4 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:42.229653 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cc507-6e9a-43b1-aa32-dbd0666771c4 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:42.229653 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cc507-6e9a-43b1-aa32-dbd0666771c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:42.248993 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-cb7cc507-6e9a-43b1-aa32-dbd0666771c4 None None] MFA Rules not processed for user `8fbc5bd64315421e97931402fb0e3a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:37:42.250202 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-cb7cc507-6e9a-43b1-aa32-dbd0666771c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:42.309704 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1225/2496] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:37:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:37:52.302514 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=25 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:37:55.380320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e2e59-c076-454d-a826-674daf5fe9a1 service swift] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:55.381879 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-bd8e2e59-c076-454d-a826-674daf5fe9a1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:55.393842 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e2e59-c076-454d-a826-674daf5fe9a1 service swift] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:55.397391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586906f9-3e49-4437-8a20-3768a7366bc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:55.398027 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586906f9-3e49-4437-8a20-3768a7366bc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:55.398310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586906f9-3e49-4437-8a20-3768a7366bc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:55.398584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586906f9-3e49-4437-8a20-3768a7366bc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:55.400283 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-586906f9-3e49-4437-8a20-3768a7366bc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:55.417464 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586906f9-3e49-4437-8a20-3768a7366bc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:55.420800 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586906f9-3e49-4437-8a20-3768a7366bc7 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:55.439158 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1272/2497] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:37:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:37:56.616095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cc507-6e9a-43b1-aa32-dbd0666771c4 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:56.626439 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7cc507-6e9a-43b1-aa32-dbd0666771c4 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:56.630034 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662840b-ad82-486c-89ce-31c7dd0620e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:56.630632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662840b-ad82-486c-89ce-31c7dd0620e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:56.630918 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662840b-ad82-486c-89ce-31c7dd0620e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:56.631190 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662840b-ad82-486c-89ce-31c7dd0620e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:56.633015 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8662840b-ad82-486c-89ce-31c7dd0620e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:56.649343 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662840b-ad82-486c-89ce-31c7dd0620e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:56.650762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662840b-ad82-486c-89ce-31c7dd0620e5 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:56.661690 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1226/2498] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:37:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:37:59.002526 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586906f9-3e49-4437-8a20-3768a7366bc7 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:37:59.005858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-586906f9-3e49-4437-8a20-3768a7366bc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:59.029762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586906f9-3e49-4437-8a20-3768a7366bc7 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:37:59.041466 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081d093-a5f5-4e07-8b4a-5565e72cace3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:37:59.042100 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081d093-a5f5-4e07-8b4a-5565e72cace3 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:37:59.042381 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081d093-a5f5-4e07-8b4a-5565e72cace3 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:37:59.042656 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081d093-a5f5-4e07-8b4a-5565e72cace3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:37:59.048142 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f081d093-a5f5-4e07-8b4a-5565e72cace3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:37:59.120255 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081d093-a5f5-4e07-8b4a-5565e72cace3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:37:59.122332 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081d093-a5f5-4e07-8b4a-5565e72cace3 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:37:59.134681 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1273/2499] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:37:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:38:03.497797 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298f1fe-4e94-4eee-ba43-e018e6305e58 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:03.498307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298f1fe-4e94-4eee-ba43-e018e6305e58 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:03.498423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298f1fe-4e94-4eee-ba43-e018e6305e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:03.518084 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e298f1fe-4e94-4eee-ba43-e018e6305e58 None None] MFA Rules not processed for user `adc5854ed52042369a9b153af7da4cfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:38:03.519338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e298f1fe-4e94-4eee-ba43-e018e6305e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:03.526528 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1227/2500] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:38:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:38:09.129349 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=34 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:38:13.512979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=9 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:38:23.545177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081d093-a5f5-4e07-8b4a-5565e72cace3 service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:23.546542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f081d093-a5f5-4e07-8b4a-5565e72cace3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:23.562841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081d093-a5f5-4e07-8b4a-5565e72cace3 service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:23.569130 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf512e-64c9-4e7a-86e7-dc25cb11a751 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:23.570118 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf512e-64c9-4e7a-86e7-dc25cb11a751 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:23.570432 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf512e-64c9-4e7a-86e7-dc25cb11a751 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:23.570801 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf512e-64c9-4e7a-86e7-dc25cb11a751 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:23.573235 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-edaf512e-64c9-4e7a-86e7-dc25cb11a751 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:23.588506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaf512e-64c9-4e7a-86e7-dc25cb11a751 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:23.591173 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaf512e-64c9-4e7a-86e7-dc25cb11a751 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:23.604175 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1274/2501] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:38:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:38:27.178082 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c932de-86d2-44ef-99b0-f0c38390e5b9 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:27.178625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c932de-86d2-44ef-99b0-f0c38390e5b9 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:27.178625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c932de-86d2-44ef-99b0-f0c38390e5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:27.198259 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-a2c932de-86d2-44ef-99b0-f0c38390e5b9 None None] MFA Rules not processed for user `068f28eb74e74650a2b80d29664109ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:38:27.199454 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a2c932de-86d2-44ef-99b0-f0c38390e5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:27.255909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1228/2502] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:38:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:38:27.570393 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf512e-64c9-4e7a-86e7-dc25cb11a751 service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:27.579725 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf512e-64c9-4e7a-86e7-dc25cb11a751 service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:27.583537 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66f9b8-8a6a-4044-befb-9a6a98b43224 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:27.584191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66f9b8-8a6a-4044-befb-9a6a98b43224 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:27.584475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66f9b8-8a6a-4044-befb-9a6a98b43224 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:27.584767 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66f9b8-8a6a-4044-befb-9a6a98b43224 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:27.586520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-8d66f9b8-8a6a-4044-befb-9a6a98b43224 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:27.596806 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66f9b8-8a6a-4044-befb-9a6a98b43224 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:27.598226 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66f9b8-8a6a-4044-befb-9a6a98b43224 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:27.606855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1275/2503] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:38:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:38:27.923698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c932de-86d2-44ef-99b0-f0c38390e5b9 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:27.925594 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a2c932de-86d2-44ef-99b0-f0c38390e5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:28.005746 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c932de-86d2-44ef-99b0-f0c38390e5b9 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:28.010706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d20d1-4a19-4cbb-9add-2366a8eda686 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0e659eee5e4a9da8d3d7337cb73798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:28.011854 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d20d1-4a19-4cbb-9add-2366a8eda686 service placement] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:28.012536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d20d1-4a19-4cbb-9add-2366a8eda686 service placement] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:28.012666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d20d1-4a19-4cbb-9add-2366a8eda686 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:28.014370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-df3d20d1-4a19-4cbb-9add-2366a8eda686 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:28.026062 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3d20d1-4a19-4cbb-9add-2366a8eda686 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:28.028332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3d20d1-4a19-4cbb-9add-2366a8eda686 service placement] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:28.038664 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1229/2504] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:38:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:38:30.451906 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66f9b8-8a6a-4044-befb-9a6a98b43224 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:30.458055 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d20d1-4a19-4cbb-9add-2366a8eda686 service placement] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:30.462070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66f9b8-8a6a-4044-befb-9a6a98b43224 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:30.469016 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024f35b-52d3-4e76-bba2-51ef8da90dc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:30.470039 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024f35b-52d3-4e76-bba2-51ef8da90dc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:30.470392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d20d1-4a19-4cbb-9add-2366a8eda686 service placement] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:30.470672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024f35b-52d3-4e76-bba2-51ef8da90dc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:30.471111 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024f35b-52d3-4e76-bba2-51ef8da90dc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:30.473337 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e024f35b-52d3-4e76-bba2-51ef8da90dc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:30.485281 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79fa76-0c46-4801-be41-c8095cb0951b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:30.485911 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79fa76-0c46-4801-be41-c8095cb0951b service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:30.486184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79fa76-0c46-4801-be41-c8095cb0951b service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:30.486549 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae79fa76-0c46-4801-be41-c8095cb0951b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:30.487279 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024f35b-52d3-4e76-bba2-51ef8da90dc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:30.489336 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024f35b-52d3-4e76-bba2-51ef8da90dc7 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:30.503325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79fa76-0c46-4801-be41-c8095cb0951b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:30.504967 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae79fa76-0c46-4801-be41-c8095cb0951b service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:30.511674 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1276/2505] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:38:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:38:30.527958 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1230/2506] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:38:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:38:34.892950 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024f35b-52d3-4e76-bba2-51ef8da90dc7 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:34.894619 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e024f35b-52d3-4e76-bba2-51ef8da90dc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:34.967123 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024f35b-52d3-4e76-bba2-51ef8da90dc7 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:34.971634 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd6423-d078-4103-965a-a12dcf0e4a7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:34.972425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd6423-d078-4103-965a-a12dcf0e4a7d service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:34.972710 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd6423-d078-4103-965a-a12dcf0e4a7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:34.972987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd6423-d078-4103-965a-a12dcf0e4a7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:34.975142 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-23bd6423-d078-4103-965a-a12dcf0e4a7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:34.988424 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd6423-d078-4103-965a-a12dcf0e4a7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:34.989884 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd6423-d078-4103-965a-a12dcf0e4a7d service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:34.999969 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1277/2507] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:38:34 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:38:40.513804 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=63 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:38:43.713834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79fa76-0c46-4801-be41-c8095cb0951b service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:43.743511 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae79fa76-0c46-4801-be41-c8095cb0951b service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:43.759437 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308336d-3246-4f9f-8626-9e53e13786f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:43.761968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308336d-3246-4f9f-8626-9e53e13786f0 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:43.763461 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308336d-3246-4f9f-8626-9e53e13786f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:43.764842 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308336d-3246-4f9f-8626-9e53e13786f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:43.769243 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f308336d-3246-4f9f-8626-9e53e13786f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:43.794262 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308336d-3246-4f9f-8626-9e53e13786f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:43.796355 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308336d-3246-4f9f-8626-9e53e13786f0 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:43.811466 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1231/2508] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:38:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:38:44.995849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=44 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:38:46.060983 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9886b82-7e81-457a-849a-1101aeb738ba None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:46.060983 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9886b82-7e81-457a-849a-1101aeb738ba None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:46.060983 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9886b82-7e81-457a-849a-1101aeb738ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:46.094439 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f9886b82-7e81-457a-849a-1101aeb738ba None None] MFA Rules not processed for user `61709ccc1e244185b252179451cd860f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:38:46.095879 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f9886b82-7e81-457a-849a-1101aeb738ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:46.104620 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1278/2509] 10.222.0.18 () {58 vars in 967 bytes} [Mon Aug 29 20:38:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:38:46.164060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308336d-3246-4f9f-8626-9e53e13786f0 service nova] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:46.175584 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308336d-3246-4f9f-8626-9e53e13786f0 service nova] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:46.183162 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05700c-8e02-4d14-9c0b-481f1d4418f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599129b7d8ff47efb145787414ddb80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:46.184347 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05700c-8e02-4d14-9c0b-481f1d4418f2 service swift] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:46.184819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05700c-8e02-4d14-9c0b-481f1d4418f2 service swift] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:46.185392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05700c-8e02-4d14-9c0b-481f1d4418f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:46.195279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05700c-8e02-4d14-9c0b-481f1d4418f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:46.196846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05700c-8e02-4d14-9c0b-481f1d4418f2 service swift] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:46.207598 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1232/2510] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 29 20:38:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:38:55.373812 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9886b82-7e81-457a-849a-1101aeb738ba None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:55.376154 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f9886b82-7e81-457a-849a-1101aeb738ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:55.485640 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9886b82-7e81-457a-849a-1101aeb738ba None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:55.490776 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142999a-3943-4337-bc84-df53e5b5e004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:55.491484 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142999a-3943-4337-bc84-df53e5b5e004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:55.492132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142999a-3943-4337-bc84-df53e5b5e004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:55.492535 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142999a-3943-4337-bc84-df53e5b5e004 admin admin] PATH_INFO: `/v3/users/448dc80e8c1b486fbdf83317209528eb` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:55.493793 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142999a-3943-4337-bc84-df53e5b5e004 admin admin] RBAC: Authorizing `identity:delete_user(user_id=448dc80e8c1b486fbdf83317209528eb)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:55.496067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142999a-3943-4337-bc84-df53e5b5e004 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:55.558820 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d142999a-3943-4337-bc84-df53e5b5e004 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '448dc80e8c1b486fbdf83317209528eb'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:55.566751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d142999a-3943-4337-bc84-df53e5b5e004 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '448dc80e8c1b486fbdf83317209528eb'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:55.579922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d142999a-3943-4337-bc84-df53e5b5e004 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '448dc80e8c1b486fbdf83317209528eb'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:55.580829 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d142999a-3943-4337-bc84-df53e5b5e004 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '448dc80e8c1b486fbdf83317209528eb'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:55.581536 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d142999a-3943-4337-bc84-df53e5b5e004 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '448dc80e8c1b486fbdf83317209528eb'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:55.601871 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1279/2511] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:38:55 2022] DELETE /identity/v3/users/448dc80e8c1b486fbdf83317209528eb => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:38:55.782076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05700c-8e02-4d14-9c0b-481f1d4418f2 service swift] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:55.794677 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05700c-8e02-4d14-9c0b-481f1d4418f2 service swift] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:55.857400 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52ab3b-12f6-4397-abb5-0987a6a9e608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:55.858041 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52ab3b-12f6-4397-abb5-0987a6a9e608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:55.858321 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52ab3b-12f6-4397-abb5-0987a6a9e608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:55.858592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52ab3b-12f6-4397-abb5-0987a6a9e608 admin admin] PATH_INFO: `/v3/projects/3f36c1186e0f44df87d13ef836f6525b` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:55.859759 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52ab3b-12f6-4397-abb5-0987a6a9e608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f36c1186e0f44df87d13ef836f6525b)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:55.861722 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c52ab3b-12f6-4397-abb5-0987a6a9e608 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:55.912497 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3c52ab3b-12f6-4397-abb5-0987a6a9e608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f36c1186e0f44df87d13ef836f6525b', 'request_id': 'req-3c52ab3b-12f6-4397-abb5-0987a6a9e608'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:55.918691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3c52ab3b-12f6-4397-abb5-0987a6a9e608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f36c1186e0f44df87d13ef836f6525b', 'request_id': 'req-3c52ab3b-12f6-4397-abb5-0987a6a9e608'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:55.928014 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3c52ab3b-12f6-4397-abb5-0987a6a9e608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f36c1186e0f44df87d13ef836f6525b', 'request_id': 'req-3c52ab3b-12f6-4397-abb5-0987a6a9e608'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:55.942698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3c52ab3b-12f6-4397-abb5-0987a6a9e608 admin admin] The token cache is being invalidate because project 3f36c1186e0f44df87d13ef836f6525b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:38:55.942985 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3c52ab3b-12f6-4397-abb5-0987a6a9e608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:55.943887 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3c52ab3b-12f6-4397-abb5-0987a6a9e608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:55.946198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1233/2512] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:38:55 2022] DELETE /identity/v3/projects/3f36c1186e0f44df87d13ef836f6525b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:38:55.950235 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142999a-3943-4337-bc84-df53e5b5e004 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:55.951710 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d142999a-3943-4337-bc84-df53e5b5e004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:55.965615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142999a-3943-4337-bc84-df53e5b5e004 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:56.027094 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89cc11-3f79-4b82-b15b-2625d931793f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:56.027736 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89cc11-3f79-4b82-b15b-2625d931793f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:56.028049 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89cc11-3f79-4b82-b15b-2625d931793f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:56.028332 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89cc11-3f79-4b82-b15b-2625d931793f admin admin] PATH_INFO: `/v3/users/4c5ce3754d6d4af8a325683564c48538` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:56.029478 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89cc11-3f79-4b82-b15b-2625d931793f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c5ce3754d6d4af8a325683564c48538)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:56.031406 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89cc11-3f79-4b82-b15b-2625d931793f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:56.045522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:38:56.080881 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3c89cc11-3f79-4b82-b15b-2625d931793f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c5ce3754d6d4af8a325683564c48538'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:56.087694 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3c89cc11-3f79-4b82-b15b-2625d931793f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c5ce3754d6d4af8a325683564c48538'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:56.095822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3c89cc11-3f79-4b82-b15b-2625d931793f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c5ce3754d6d4af8a325683564c48538'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:56.096570 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3c89cc11-3f79-4b82-b15b-2625d931793f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c5ce3754d6d4af8a325683564c48538'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:56.097193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-3c89cc11-3f79-4b82-b15b-2625d931793f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c5ce3754d6d4af8a325683564c48538'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:56.114838 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1280/2513] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:38:55 2022] DELETE /identity/v3/users/4c5ce3754d6d4af8a325683564c48538 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:38:56.288271 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52ab3b-12f6-4397-abb5-0987a6a9e608 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:38:56.290116 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-3c52ab3b-12f6-4397-abb5-0987a6a9e608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:38:56.310321 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52ab3b-12f6-4397-abb5-0987a6a9e608 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:38:56.371653 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bf12e-32d8-41cf-a4b2-beacdebdb80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:38:56.372327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bf12e-32d8-41cf-a4b2-beacdebdb80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:38:56.372607 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bf12e-32d8-41cf-a4b2-beacdebdb80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:38:56.372879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bf12e-32d8-41cf-a4b2-beacdebdb80a admin admin] PATH_INFO: `/v3/projects/fd7f1416e786402898e6377e1c3f210c` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:38:56.374007 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34bf12e-32d8-41cf-a4b2-beacdebdb80a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd7f1416e786402898e6377e1c3f210c)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:38:56.376041 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34bf12e-32d8-41cf-a4b2-beacdebdb80a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:38:56.382315 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:38:56.424358 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d34bf12e-32d8-41cf-a4b2-beacdebdb80a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd7f1416e786402898e6377e1c3f210c', 'request_id': 'req-d34bf12e-32d8-41cf-a4b2-beacdebdb80a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:56.429750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d34bf12e-32d8-41cf-a4b2-beacdebdb80a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd7f1416e786402898e6377e1c3f210c', 'request_id': 'req-d34bf12e-32d8-41cf-a4b2-beacdebdb80a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:56.437975 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d34bf12e-32d8-41cf-a4b2-beacdebdb80a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd7f1416e786402898e6377e1c3f210c', 'request_id': 'req-d34bf12e-32d8-41cf-a4b2-beacdebdb80a'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:56.449877 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d34bf12e-32d8-41cf-a4b2-beacdebdb80a admin admin] The token cache is being invalidate because project fd7f1416e786402898e6377e1c3f210c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:38:56.450211 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d34bf12e-32d8-41cf-a4b2-beacdebdb80a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:56.450916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d34bf12e-32d8-41cf-a4b2-beacdebdb80a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:38:56.452841 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1234/2514] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:38:56 2022] DELETE /identity/v3/projects/fd7f1416e786402898e6377e1c3f210c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:03.349467 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89cc11-3f79-4b82-b15b-2625d931793f admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:03.351049 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3c89cc11-3f79-4b82-b15b-2625d931793f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:03.461318 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89cc11-3f79-4b82-b15b-2625d931793f admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:03.531117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504d8c8-1fad-4173-a94c-f3b76bc24a31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:03.531736 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504d8c8-1fad-4173-a94c-f3b76bc24a31 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:03.532043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504d8c8-1fad-4173-a94c-f3b76bc24a31 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:03.532317 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504d8c8-1fad-4173-a94c-f3b76bc24a31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:03.534106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d504d8c8-1fad-4173-a94c-f3b76bc24a31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:03.546578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504d8c8-1fad-4173-a94c-f3b76bc24a31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:03.548194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504d8c8-1fad-4173-a94c-f3b76bc24a31 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:03.561322 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1281/2515] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:03.594227 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bf12e-32d8-41cf-a4b2-beacdebdb80a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:03.595685 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d34bf12e-32d8-41cf-a4b2-beacdebdb80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:03.657253 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bf12e-32d8-41cf-a4b2-beacdebdb80a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:03.660902 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de1772-b550-41f9-940e-069a807c33af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:03.661551 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de1772-b550-41f9-940e-069a807c33af service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:03.661792 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de1772-b550-41f9-940e-069a807c33af service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:03.662064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de1772-b550-41f9-940e-069a807c33af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:03.663777 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-99de1772-b550-41f9-940e-069a807c33af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:03.728658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de1772-b550-41f9-940e-069a807c33af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:03.730086 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99de1772-b550-41f9-940e-069a807c33af service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:03.739996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1235/2516] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:04.391049 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f191f1d-947e-4f80-b8b7-bd77b4189e04 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:04.391598 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f191f1d-947e-4f80-b8b7-bd77b4189e04 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:04.391820 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f191f1d-947e-4f80-b8b7-bd77b4189e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:04.413338 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-0f191f1d-947e-4f80-b8b7-bd77b4189e04 None None] MFA Rules not processed for user `fca06f41184444ccb0ddb57e5d44f227`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:04.414678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0f191f1d-947e-4f80-b8b7-bd77b4189e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:04.427149 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:04.504246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1282/2517] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:04.632715 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8e22f-52bc-4cf9-b40c-f8cab5e81318 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:04.633261 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8e22f-52bc-4cf9-b40c-f8cab5e81318 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:04.633261 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8e22f-52bc-4cf9-b40c-f8cab5e81318 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:04.635098 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1236/2518] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 29 20:39:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:39:04.642171 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615ae3e-faaa-48d1-acb6-630ddb63bba4 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:04.642458 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615ae3e-faaa-48d1-acb6-630ddb63bba4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:04.642952 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615ae3e-faaa-48d1-acb6-630ddb63bba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:04.671402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-6615ae3e-faaa-48d1-acb6-630ddb63bba4 None None] MFA Rules not processed for user `d51e331a4c9c4f038bdb80f8c5b74e42`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:04.672665 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6615ae3e-faaa-48d1-acb6-630ddb63bba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:04.679666 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1283/2519] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 29 20:39:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:39:04.712495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8e22f-52bc-4cf9-b40c-f8cab5e81318 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:04.726319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8e22f-52bc-4cf9-b40c-f8cab5e81318 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:04.731462 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1841d0-c799-4b1f-8b67-87847ff709e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:04.732578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1841d0-c799-4b1f-8b67-87847ff709e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:04.733023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1841d0-c799-4b1f-8b67-87847ff709e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:04.733465 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1841d0-c799-4b1f-8b67-87847ff709e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:04.750045 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1841d0-c799-4b1f-8b67-87847ff709e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:04.751454 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1841d0-c799-4b1f-8b67-87847ff709e1 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:04.760183 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1237/2520] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:04.892645 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615ae3e-faaa-48d1-acb6-630ddb63bba4 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:04.894051 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6615ae3e-faaa-48d1-acb6-630ddb63bba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:04.919168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615ae3e-faaa-48d1-acb6-630ddb63bba4 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:04.930490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f0312-785a-4250-a37b-df973a347e76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:04.931484 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f0312-785a-4250-a37b-df973a347e76 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:04.931761 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f0312-785a-4250-a37b-df973a347e76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:04.932080 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f0312-785a-4250-a37b-df973a347e76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:04.933964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2c3f0312-785a-4250-a37b-df973a347e76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:04.944549 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08bb71f-da9c-48eb-9370-2aa81608aa6f None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:04.945421 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08bb71f-da9c-48eb-9370-2aa81608aa6f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:04.945769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08bb71f-da9c-48eb-9370-2aa81608aa6f None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:04.946007 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f0312-785a-4250-a37b-df973a347e76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:04.947429 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1238/2521] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 29 20:39:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 29 20:39:04.947551 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f0312-785a-4250-a37b-df973a347e76 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:04.954234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f1718-4237-48ac-be6e-1b3b45eea351 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:04.954580 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f1718-4237-48ac-be6e-1b3b45eea351 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:04.954926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f1718-4237-48ac-be6e-1b3b45eea351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:04.956622 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1284/2522] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:39:04.977906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-116f1718-4237-48ac-be6e-1b3b45eea351 None None] MFA Rules not processed for user `c0c44754a1c04a2b87081aba6cb509d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:04.979885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-116f1718-4237-48ac-be6e-1b3b45eea351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:04.990293 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1239/2523] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 29 20:39:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:39:05.012652 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f0312-785a-4250-a37b-df973a347e76 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:05.024268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f0312-785a-4250-a37b-df973a347e76 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:05.032428 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00aa5ad-8a96-4e6b-8d08-6a850df0bce8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:05.033257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00aa5ad-8a96-4e6b-8d08-6a850df0bce8 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:05.033706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00aa5ad-8a96-4e6b-8d08-6a850df0bce8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:05.034322 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00aa5ad-8a96-4e6b-8d08-6a850df0bce8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:05.057173 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00aa5ad-8a96-4e6b-8d08-6a850df0bce8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:05.059806 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00aa5ad-8a96-4e6b-8d08-6a850df0bce8 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:05.078563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1285/2524] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:08.255010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f1718-4237-48ac-be6e-1b3b45eea351 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:08.256488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-116f1718-4237-48ac-be6e-1b3b45eea351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:08.297004 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:08.336376 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00aa5ad-8a96-4e6b-8d08-6a850df0bce8 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:08.338013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a00aa5ad-8a96-4e6b-8d08-6a850df0bce8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:08.355420 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00aa5ad-8a96-4e6b-8d08-6a850df0bce8 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:08.356884 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f1718-4237-48ac-be6e-1b3b45eea351 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:08.361040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78867f1-e229-4664-96bd-9ad4e118c9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:08.361805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78867f1-e229-4664-96bd-9ad4e118c9fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:08.362090 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78867f1-e229-4664-96bd-9ad4e118c9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:08.362366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78867f1-e229-4664-96bd-9ad4e118c9fa admin admin] PATH_INFO: `/v3/users/8fbc5bd64315421e97931402fb0e3a6a` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:08.362515 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83251bfb-3037-494c-b461-f3a508da2580 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0e659eee5e4a9da8d3d7337cb73798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:08.363140 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83251bfb-3037-494c-b461-f3a508da2580 service placement] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:08.363518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83251bfb-3037-494c-b461-f3a508da2580 service placement] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:08.363798 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83251bfb-3037-494c-b461-f3a508da2580 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:08.364017 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78867f1-e229-4664-96bd-9ad4e118c9fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fbc5bd64315421e97931402fb0e3a6a)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:08.365616 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-83251bfb-3037-494c-b461-f3a508da2580 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:08.366218 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78867f1-e229-4664-96bd-9ad4e118c9fa admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:08.379972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83251bfb-3037-494c-b461-f3a508da2580 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:08.381738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83251bfb-3037-494c-b461-f3a508da2580 service placement] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:08.393790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1240/2525] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:08.438330 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c78867f1-e229-4664-96bd-9ad4e118c9fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fbc5bd64315421e97931402fb0e3a6a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:08.446615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c78867f1-e229-4664-96bd-9ad4e118c9fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fbc5bd64315421e97931402fb0e3a6a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:08.454153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c78867f1-e229-4664-96bd-9ad4e118c9fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fbc5bd64315421e97931402fb0e3a6a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:08.454874 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c78867f1-e229-4664-96bd-9ad4e118c9fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fbc5bd64315421e97931402fb0e3a6a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:08.455578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c78867f1-e229-4664-96bd-9ad4e118c9fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fbc5bd64315421e97931402fb0e3a6a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:08.475077 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1286/2526] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:08 2022] DELETE /identity/v3/users/8fbc5bd64315421e97931402fb0e3a6a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:08.660677 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83251bfb-3037-494c-b461-f3a508da2580 service placement] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:08.675558 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83251bfb-3037-494c-b461-f3a508da2580 service placement] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:08.747373 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d04fea-d7dd-4569-aa26-5a693aad369f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:08.748031 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d04fea-d7dd-4569-aa26-5a693aad369f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:08.748341 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d04fea-d7dd-4569-aa26-5a693aad369f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:08.748614 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d04fea-d7dd-4569-aa26-5a693aad369f admin admin] PATH_INFO: `/v3/projects/8436d85d931a41cfa6d626c08ad82ef4` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:08.749737 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d04fea-d7dd-4569-aa26-5a693aad369f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8436d85d931a41cfa6d626c08ad82ef4)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:08.751654 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d04fea-d7dd-4569-aa26-5a693aad369f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:08.803348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-82d04fea-d7dd-4569-aa26-5a693aad369f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8436d85d931a41cfa6d626c08ad82ef4', 'request_id': 'req-82d04fea-d7dd-4569-aa26-5a693aad369f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:08.810949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-82d04fea-d7dd-4569-aa26-5a693aad369f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8436d85d931a41cfa6d626c08ad82ef4', 'request_id': 'req-82d04fea-d7dd-4569-aa26-5a693aad369f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:08.821094 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-82d04fea-d7dd-4569-aa26-5a693aad369f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8436d85d931a41cfa6d626c08ad82ef4', 'request_id': 'req-82d04fea-d7dd-4569-aa26-5a693aad369f'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:08.839428 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-82d04fea-d7dd-4569-aa26-5a693aad369f admin admin] The token cache is being invalidate because project 8436d85d931a41cfa6d626c08ad82ef4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:39:08.839718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-82d04fea-d7dd-4569-aa26-5a693aad369f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:08.840419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-82d04fea-d7dd-4569-aa26-5a693aad369f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:08.841918 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1241/2527] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:08 2022] DELETE /identity/v3/projects/8436d85d931a41cfa6d626c08ad82ef4 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:08.846798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee314504-f958-4a2b-8b5e-076915cfb491 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:08.847188 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee314504-f958-4a2b-8b5e-076915cfb491 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:08.847563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee314504-f958-4a2b-8b5e-076915cfb491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:08.877043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ee314504-f958-4a2b-8b5e-076915cfb491 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:08.878474 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ee314504-f958-4a2b-8b5e-076915cfb491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:08.944049 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:08.955190 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1287/2528] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:08.963955 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee65546-3312-4e50-83d1-c7b2bad27cda None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:08.964319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee65546-3312-4e50-83d1-c7b2bad27cda None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:08.964595 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee65546-3312-4e50-83d1-c7b2bad27cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:08.984875 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-bee65546-3312-4e50-83d1-c7b2bad27cda None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:08.986078 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bee65546-3312-4e50-83d1-c7b2bad27cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:08.994134 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1242/2529] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:08.998503 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee314504-f958-4a2b-8b5e-076915cfb491 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:09.008803 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee314504-f958-4a2b-8b5e-076915cfb491 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:09.013992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caafaf99-c91d-4a73-8aa3-ab219fd2517f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:09.014614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafaf99-c91d-4a73-8aa3-ab219fd2517f admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:09.014913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafaf99-c91d-4a73-8aa3-ab219fd2517f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:09.015187 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caafaf99-c91d-4a73-8aa3-ab219fd2517f admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:09.015863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caafaf99-c91d-4a73-8aa3-ab219fd2517f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:39:09.016265 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caafaf99-c91d-4a73-8aa3-ab219fd2517f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:09.017906 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caafaf99-c91d-4a73-8aa3-ab219fd2517f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:09.030871 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1288/2530] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:39:09.035365 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee65546-3312-4e50-83d1-c7b2bad27cda None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:09.047115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee65546-3312-4e50-83d1-c7b2bad27cda None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:09.054779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec32737-2692-477e-bf6a-6bd58d329a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:09.055666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec32737-2692-477e-bf6a-6bd58d329a2a admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:09.056192 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec32737-2692-477e-bf6a-6bd58d329a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:09.056964 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec32737-2692-477e-bf6a-6bd58d329a2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:09.057694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec32737-2692-477e-bf6a-6bd58d329a2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:09.061494 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec32737-2692-477e-bf6a-6bd58d329a2a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:09.110006 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1243/2531] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:09 2022] POST /identity/v3/projects => generated 365 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:09.112590 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caafaf99-c91d-4a73-8aa3-ab219fd2517f admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:09.133663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caafaf99-c91d-4a73-8aa3-ab219fd2517f admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:09.205208 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0037e6f-b610-410a-bca8-db9468696b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:09.205834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0037e6f-b610-410a-bca8-db9468696b49 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:09.206116 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0037e6f-b610-410a-bca8-db9468696b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:09.206388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0037e6f-b610-410a-bca8-db9468696b49 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:09.207040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0037e6f-b610-410a-bca8-db9468696b49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:09.209035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0037e6f-b610-410a-bca8-db9468696b49 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:09.232618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1289/2532] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:09 2022] POST /identity/v3/users => generated 283 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:09.236758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec32737-2692-477e-bf6a-6bd58d329a2a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:09.247882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec32737-2692-477e-bf6a-6bd58d329a2a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:09.252685 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef842e-81e0-4f80-a479-83e04fe976ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:09.253287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef842e-81e0-4f80-a479-83e04fe976ee admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:09.253566 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef842e-81e0-4f80-a479-83e04fe976ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:09.253839 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef842e-81e0-4f80-a479-83e04fe976ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:09.254505 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef842e-81e0-4f80-a479-83e04fe976ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:09.257038 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ef842e-81e0-4f80-a479-83e04fe976ee admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:09.268596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1244/2533] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:09.272575 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0037e6f-b610-410a-bca8-db9468696b49 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:09.283866 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0037e6f-b610-410a-bca8-db9468696b49 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:09.289260 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11978c68-990e-49b9-b553-b2cf36b3e694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:09.289898 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11978c68-990e-49b9-b553-b2cf36b3e694 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:09.290183 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11978c68-990e-49b9-b553-b2cf36b3e694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:09.290632 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11978c68-990e-49b9-b553-b2cf36b3e694 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:09.291307 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11978c68-990e-49b9-b553-b2cf36b3e694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:09.293116 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11978c68-990e-49b9-b553-b2cf36b3e694 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:09.307489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1290/2534] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:09.309136 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef842e-81e0-4f80-a479-83e04fe976ee admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:09.316598 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:09.320814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef842e-81e0-4f80-a479-83e04fe976ee admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:09.328295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebf087-ce62-4a87-836b-1a716c3be10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:09.329325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebf087-ce62-4a87-836b-1a716c3be10c admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:09.329508 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebf087-ce62-4a87-836b-1a716c3be10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:09.329848 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebf087-ce62-4a87-836b-1a716c3be10c admin admin] PATH_INFO: `/v3/projects/b6b8bf72129f403caca1d084504db7a9/users/01e7f86064a0412da8cead57529806f6/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:09.346475 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebf087-ce62-4a87-836b-1a716c3be10c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6b8bf72129f403caca1d084504db7a9, user_id=01e7f86064a0412da8cead57529806f6, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:09.349099 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebf087-ce62-4a87-836b-1a716c3be10c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:09.363001 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1245/2535] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:09 2022] PUT /identity/v3/projects/b6b8bf72129f403caca1d084504db7a9/users/01e7f86064a0412da8cead57529806f6/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:39:09.374412 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97848663-228a-4384-b087-8e941a1102ee None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:09.374976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97848663-228a-4384-b087-8e941a1102ee None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:09.374976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97848663-228a-4384-b087-8e941a1102ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:09.404547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-97848663-228a-4384-b087-8e941a1102ee None None] MFA Rules not processed for user `01e7f86064a0412da8cead57529806f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:09.406340 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-97848663-228a-4384-b087-8e941a1102ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:09.479473 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1291/2536] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:09.484016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebf087-ce62-4a87-836b-1a716c3be10c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:09.561565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebf087-ce62-4a87-836b-1a716c3be10c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:09.566623 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38a7ad-e947-4518-b19f-3b8bfa26dc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:09.567164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38a7ad-e947-4518-b19f-3b8bfa26dc0f admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:09.567446 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38a7ad-e947-4518-b19f-3b8bfa26dc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:09.567724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38a7ad-e947-4518-b19f-3b8bfa26dc0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:09.568422 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38a7ad-e947-4518-b19f-3b8bfa26dc0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:09.570350 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38a7ad-e947-4518-b19f-3b8bfa26dc0f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:09.601877 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1246/2537] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:09 2022] POST /identity/v3/projects => generated 363 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:09.603126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97848663-228a-4384-b087-8e941a1102ee None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:09.614535 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97848663-228a-4384-b087-8e941a1102ee None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:09.666346 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:09.674974 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe9d85-59d9-4cef-9472-03772b8cad1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:09.675627 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe9d85-59d9-4cef-9472-03772b8cad1e admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:09.676067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe9d85-59d9-4cef-9472-03772b8cad1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:09.676215 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe9d85-59d9-4cef-9472-03772b8cad1e admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:09.676858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe9d85-59d9-4cef-9472-03772b8cad1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:09.678890 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe9d85-59d9-4cef-9472-03772b8cad1e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:09.704007 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1292/2538] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:09 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:09.706309 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38a7ad-e947-4518-b19f-3b8bfa26dc0f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:09.717120 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38a7ad-e947-4518-b19f-3b8bfa26dc0f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:09.722480 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a09228-aacb-43c5-ae70-2ba5048bab26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:09.723093 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a09228-aacb-43c5-ae70-2ba5048bab26 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:09.723373 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a09228-aacb-43c5-ae70-2ba5048bab26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:09.723645 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a09228-aacb-43c5-ae70-2ba5048bab26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:09.724557 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a09228-aacb-43c5-ae70-2ba5048bab26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:09.725938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a09228-aacb-43c5-ae70-2ba5048bab26 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:09.741334 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1247/2539] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:09.742676 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe9d85-59d9-4cef-9472-03772b8cad1e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:09.753538 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe9d85-59d9-4cef-9472-03772b8cad1e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:09.758209 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2b0a5-0bd2-4f28-aaea-fdbb87778ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:09.758880 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2b0a5-0bd2-4f28-aaea-fdbb87778ebf admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:09.759164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2b0a5-0bd2-4f28-aaea-fdbb87778ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:09.760044 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2b0a5-0bd2-4f28-aaea-fdbb87778ebf admin admin] PATH_INFO: `/v3/projects/23155237eaf84a6280895dcc7120cebd/users/d4582c3e3998445fbe67ec2bf11ba283/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:09.776336 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2b0a5-0bd2-4f28-aaea-fdbb87778ebf admin admin] RBAC: Authorizing `identity:create_grant(project_id=23155237eaf84a6280895dcc7120cebd, user_id=d4582c3e3998445fbe67ec2bf11ba283, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:09.779178 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2b0a5-0bd2-4f28-aaea-fdbb87778ebf admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:09.793749 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1293/2540] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:09 2022] PUT /identity/v3/projects/23155237eaf84a6280895dcc7120cebd/users/d4582c3e3998445fbe67ec2bf11ba283/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:09.797816 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a09228-aacb-43c5-ae70-2ba5048bab26 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:09.808871 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a09228-aacb-43c5-ae70-2ba5048bab26 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:09.869914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714c584-2ac0-4018-8745-35d42fde021f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:09.870888 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714c584-2ac0-4018-8745-35d42fde021f admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:09.871345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714c584-2ac0-4018-8745-35d42fde021f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:09.871760 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714c584-2ac0-4018-8745-35d42fde021f admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:09.872743 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7714c584-2ac0-4018-8745-35d42fde021f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:09.874616 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7714c584-2ac0-4018-8745-35d42fde021f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:09.889378 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1248/2541] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:39:09.893196 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2b0a5-0bd2-4f28-aaea-fdbb87778ebf admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:09.904678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2b0a5-0bd2-4f28-aaea-fdbb87778ebf admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:09.909500 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3f712-7c9f-493b-8e97-810d10c9a400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:09.910202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3f712-7c9f-493b-8e97-810d10c9a400 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:09.910474 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3f712-7c9f-493b-8e97-810d10c9a400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:09.910751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3f712-7c9f-493b-8e97-810d10c9a400 admin admin] PATH_INFO: `/v3/projects/23155237eaf84a6280895dcc7120cebd/users/d4582c3e3998445fbe67ec2bf11ba283/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:09.912883 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3f712-7c9f-493b-8e97-810d10c9a400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23155237eaf84a6280895dcc7120cebd, user_id=d4582c3e3998445fbe67ec2bf11ba283, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:09.916103 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3f712-7c9f-493b-8e97-810d10c9a400 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:09.929280 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1294/2542] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:09 2022] PUT /identity/v3/projects/23155237eaf84a6280895dcc7120cebd/users/d4582c3e3998445fbe67ec2bf11ba283/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:39:09.942745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da18d3a-3777-4add-9be1-50d58dbb6dbf None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:09.942745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da18d3a-3777-4add-9be1-50d58dbb6dbf None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:09.943069 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da18d3a-3777-4add-9be1-50d58dbb6dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:09.973595 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-0da18d3a-3777-4add-9be1-50d58dbb6dbf None None] MFA Rules not processed for user `d4582c3e3998445fbe67ec2bf11ba283`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:09.974994 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0da18d3a-3777-4add-9be1-50d58dbb6dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:10.028130 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:10.068534 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1249/2543] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:09 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:10.081818 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3f712-7c9f-493b-8e97-810d10c9a400 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:10.090871 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-1cf3f712-7c9f-493b-8e97-810d10c9a400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:10.161736 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3f712-7c9f-493b-8e97-810d10c9a400 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:10.166564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44896bc2-5a21-4074-912e-c4bebb4af21d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:10.167020 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44896bc2-5a21-4074-912e-c4bebb4af21d service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:10.167311 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44896bc2-5a21-4074-912e-c4bebb4af21d service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:10.167595 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44896bc2-5a21-4074-912e-c4bebb4af21d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:10.179557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44896bc2-5a21-4074-912e-c4bebb4af21d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:10.181140 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44896bc2-5a21-4074-912e-c4bebb4af21d service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:10.191722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1295/2544] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:10 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:10.655289 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da18d3a-3777-4add-9be1-50d58dbb6dbf None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:10.720390 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da18d3a-3777-4add-9be1-50d58dbb6dbf None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:10.725142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784beaf-8e5d-4793-af83-7c62024fd75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:10.725772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784beaf-8e5d-4793-af83-7c62024fd75a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:10.726050 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784beaf-8e5d-4793-af83-7c62024fd75a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:10.726321 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784beaf-8e5d-4793-af83-7c62024fd75a admin admin] PATH_INFO: `/v3/users/01e7f86064a0412da8cead57529806f6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:10.727447 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784beaf-8e5d-4793-af83-7c62024fd75a admin admin] RBAC: Authorizing `identity:delete_user(user_id=01e7f86064a0412da8cead57529806f6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:10.729902 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784beaf-8e5d-4793-af83-7c62024fd75a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:10.776755 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7784beaf-8e5d-4793-af83-7c62024fd75a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01e7f86064a0412da8cead57529806f6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:10.784131 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7784beaf-8e5d-4793-af83-7c62024fd75a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01e7f86064a0412da8cead57529806f6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:10.792126 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7784beaf-8e5d-4793-af83-7c62024fd75a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01e7f86064a0412da8cead57529806f6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:10.792739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7784beaf-8e5d-4793-af83-7c62024fd75a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01e7f86064a0412da8cead57529806f6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:10.804806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-7784beaf-8e5d-4793-af83-7c62024fd75a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01e7f86064a0412da8cead57529806f6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:10.812141 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1250/2545] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:10 2022] DELETE /identity/v3/users/01e7f86064a0412da8cead57529806f6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:10.816825 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ebf85-6a40-4c75-8083-dd350007a128 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:10.817316 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ebf85-6a40-4c75-8083-dd350007a128 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:10.817435 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ebf85-6a40-4c75-8083-dd350007a128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:10.837352 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-096ebf85-6a40-4c75-8083-dd350007a128 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:10.838551 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-096ebf85-6a40-4c75-8083-dd350007a128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:10.898490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1296/2546] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:10.909132 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784beaf-8e5d-4793-af83-7c62024fd75a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:10.910516 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7784beaf-8e5d-4793-af83-7c62024fd75a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:10.968158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784beaf-8e5d-4793-af83-7c62024fd75a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:10.971717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11152519-6905-4f2d-805c-e911e57ebc31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:10.972344 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11152519-6905-4f2d-805c-e911e57ebc31 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:10.972617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11152519-6905-4f2d-805c-e911e57ebc31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:10.972887 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11152519-6905-4f2d-805c-e911e57ebc31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:10.982272 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11152519-6905-4f2d-805c-e911e57ebc31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:10.983686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11152519-6905-4f2d-805c-e911e57ebc31 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:10.987546 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:10.994799 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1251/2547] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:11.308834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ebf85-6a40-4c75-8083-dd350007a128 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:11.310793 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-096ebf85-6a40-4c75-8083-dd350007a128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:11.326764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ebf85-6a40-4c75-8083-dd350007a128 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:11.331522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717085f-c8a4-40e9-945e-561880e86f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:11.332223 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717085f-c8a4-40e9-945e-561880e86f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:11.332514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717085f-c8a4-40e9-945e-561880e86f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:11.333040 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717085f-c8a4-40e9-945e-561880e86f6d admin admin] PATH_INFO: `/v3/projects/b6b8bf72129f403caca1d084504db7a9` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:11.333996 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717085f-c8a4-40e9-945e-561880e86f6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6b8bf72129f403caca1d084504db7a9)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:11.335890 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717085f-c8a4-40e9-945e-561880e86f6d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:11.383387 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:11.387124 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a717085f-c8a4-40e9-945e-561880e86f6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6b8bf72129f403caca1d084504db7a9', 'request_id': 'req-a717085f-c8a4-40e9-945e-561880e86f6d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:11.396093 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a717085f-c8a4-40e9-945e-561880e86f6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6b8bf72129f403caca1d084504db7a9', 'request_id': 'req-a717085f-c8a4-40e9-945e-561880e86f6d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:11.402434 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a717085f-c8a4-40e9-945e-561880e86f6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6b8bf72129f403caca1d084504db7a9', 'request_id': 'req-a717085f-c8a4-40e9-945e-561880e86f6d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:11.414074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a717085f-c8a4-40e9-945e-561880e86f6d admin admin] The token cache is being invalidate because project b6b8bf72129f403caca1d084504db7a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:39:11.414364 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a717085f-c8a4-40e9-945e-561880e86f6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:11.415016 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-a717085f-c8a4-40e9-945e-561880e86f6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:11.416530 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1297/2548] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:11 2022] DELETE /identity/v3/projects/b6b8bf72129f403caca1d084504db7a9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:11.420520 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11152519-6905-4f2d-805c-e911e57ebc31 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:11.430171 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11152519-6905-4f2d-805c-e911e57ebc31 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:11.486854 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab252e-7c0e-4829-93f4-ef98c07cd057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:11.487509 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab252e-7c0e-4829-93f4-ef98c07cd057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:11.487783 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab252e-7c0e-4829-93f4-ef98c07cd057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:11.488085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab252e-7c0e-4829-93f4-ef98c07cd057 admin admin] PATH_INFO: `/v3/users/d4582c3e3998445fbe67ec2bf11ba283` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:11.489199 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab252e-7c0e-4829-93f4-ef98c07cd057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4582c3e3998445fbe67ec2bf11ba283)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:11.491075 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab252e-7c0e-4829-93f4-ef98c07cd057 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:11.536307 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-06ab252e-7c0e-4829-93f4-ef98c07cd057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4582c3e3998445fbe67ec2bf11ba283'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:11.544706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-06ab252e-7c0e-4829-93f4-ef98c07cd057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4582c3e3998445fbe67ec2bf11ba283'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:11.550314 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-06ab252e-7c0e-4829-93f4-ef98c07cd057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4582c3e3998445fbe67ec2bf11ba283'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:11.550942 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-06ab252e-7c0e-4829-93f4-ef98c07cd057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4582c3e3998445fbe67ec2bf11ba283'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:11.563896 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-06ab252e-7c0e-4829-93f4-ef98c07cd057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4582c3e3998445fbe67ec2bf11ba283'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:11.571130 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1252/2549] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:11 2022] DELETE /identity/v3/users/d4582c3e3998445fbe67ec2bf11ba283 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:39:11.860979 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717085f-c8a4-40e9-945e-561880e86f6d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:11.862450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a717085f-c8a4-40e9-945e-561880e86f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:11.877515 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717085f-c8a4-40e9-945e-561880e86f6d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:11.938751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f3da1-4cdd-4806-9245-ad2b0dc72337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:11.939405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f3da1-4cdd-4806-9245-ad2b0dc72337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:11.939765 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f3da1-4cdd-4806-9245-ad2b0dc72337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:11.940079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f3da1-4cdd-4806-9245-ad2b0dc72337 admin admin] PATH_INFO: `/v3/projects/23155237eaf84a6280895dcc7120cebd` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:11.941210 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f3da1-4cdd-4806-9245-ad2b0dc72337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23155237eaf84a6280895dcc7120cebd)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:11.943168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f3da1-4cdd-4806-9245-ad2b0dc72337 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:11.989570 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0c1f3da1-4cdd-4806-9245-ad2b0dc72337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23155237eaf84a6280895dcc7120cebd', 'request_id': 'req-0c1f3da1-4cdd-4806-9245-ad2b0dc72337'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:11.998031 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0c1f3da1-4cdd-4806-9245-ad2b0dc72337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23155237eaf84a6280895dcc7120cebd', 'request_id': 'req-0c1f3da1-4cdd-4806-9245-ad2b0dc72337'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:12.003761 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0c1f3da1-4cdd-4806-9245-ad2b0dc72337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23155237eaf84a6280895dcc7120cebd', 'request_id': 'req-0c1f3da1-4cdd-4806-9245-ad2b0dc72337'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:12.016199 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0c1f3da1-4cdd-4806-9245-ad2b0dc72337 admin admin] The token cache is being invalidate because project 23155237eaf84a6280895dcc7120cebd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:39:12.016495 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0c1f3da1-4cdd-4806-9245-ad2b0dc72337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:12.017144 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0c1f3da1-4cdd-4806-9245-ad2b0dc72337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:12.018631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1298/2550] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:11 2022] DELETE /identity/v3/projects/23155237eaf84a6280895dcc7120cebd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:12.023544 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44ebeb-596d-4bea-bfa6-f414832534ce None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:12.024088 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44ebeb-596d-4bea-bfa6-f414832534ce None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:12.024202 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44ebeb-596d-4bea-bfa6-f414832534ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:12.043841 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-8a44ebeb-596d-4bea-bfa6-f414832534ce None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:12.045138 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8a44ebeb-596d-4bea-bfa6-f414832534ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:12.109799 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1253/2551] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:12.117862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ace988-4f48-490e-8faf-b70139aa9347 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:12.118148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ace988-4f48-490e-8faf-b70139aa9347 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:12.118423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ace988-4f48-490e-8faf-b70139aa9347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:12.141886 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-45ace988-4f48-490e-8faf-b70139aa9347 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:12.143099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-45ace988-4f48-490e-8faf-b70139aa9347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:12.151019 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1299/2552] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:12.155328 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44ebeb-596d-4bea-bfa6-f414832534ce None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:12.166869 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44ebeb-596d-4bea-bfa6-f414832534ce None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:12.172175 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d06ec5-53cc-4655-b21e-a53eba2b95e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:12.172817 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d06ec5-53cc-4655-b21e-a53eba2b95e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:12.173098 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d06ec5-53cc-4655-b21e-a53eba2b95e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:12.173372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d06ec5-53cc-4655-b21e-a53eba2b95e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:12.174044 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d06ec5-53cc-4655-b21e-a53eba2b95e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:39:12.174468 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d06ec5-53cc-4655-b21e-a53eba2b95e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:12.176124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d06ec5-53cc-4655-b21e-a53eba2b95e8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:12.189550 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1254/2553] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:39:12.194142 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ace988-4f48-490e-8faf-b70139aa9347 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:12.206963 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ace988-4f48-490e-8faf-b70139aa9347 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:12.215058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805280e-ed4b-4a40-8226-a4e5d1a65153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:12.216514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805280e-ed4b-4a40-8226-a4e5d1a65153 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:12.216806 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805280e-ed4b-4a40-8226-a4e5d1a65153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:12.217083 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805280e-ed4b-4a40-8226-a4e5d1a65153 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:12.218350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805280e-ed4b-4a40-8226-a4e5d1a65153 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:12.220650 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805280e-ed4b-4a40-8226-a4e5d1a65153 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:12.257872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1300/2554] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:12 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:12.260619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d06ec5-53cc-4655-b21e-a53eba2b95e8 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:12.267295 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:12.274301 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d06ec5-53cc-4655-b21e-a53eba2b95e8 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:12.397809 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c358b-d5a5-4ab0-b832-3f02331a1ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:12.398520 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c358b-d5a5-4ab0-b832-3f02331a1ccb admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:12.398908 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c358b-d5a5-4ab0-b832-3f02331a1ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:12.399215 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c358b-d5a5-4ab0-b832-3f02331a1ccb admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:12.400519 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c358b-d5a5-4ab0-b832-3f02331a1ccb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:12.402639 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c358b-d5a5-4ab0-b832-3f02331a1ccb admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:12.431971 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1255/2555] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:12 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:39:12.436567 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805280e-ed4b-4a40-8226-a4e5d1a65153 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:12.448163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805280e-ed4b-4a40-8226-a4e5d1a65153 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:12.453648 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9e343-5014-4e74-8fd9-4ba003fce96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:12.454433 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9e343-5014-4e74-8fd9-4ba003fce96c admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:12.454783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9e343-5014-4e74-8fd9-4ba003fce96c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:12.455121 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9e343-5014-4e74-8fd9-4ba003fce96c admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:12.455894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9e343-5014-4e74-8fd9-4ba003fce96c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:12.457592 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9e343-5014-4e74-8fd9-4ba003fce96c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:12.473316 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1301/2556] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:12.477537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c358b-d5a5-4ab0-b832-3f02331a1ccb admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:12.491387 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c358b-d5a5-4ab0-b832-3f02331a1ccb admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:12.496440 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c38207-5301-4a1a-a146-a9dc0890154a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:12.497048 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c38207-5301-4a1a-a146-a9dc0890154a admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:12.497334 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c38207-5301-4a1a-a146-a9dc0890154a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:12.497609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c38207-5301-4a1a-a146-a9dc0890154a admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:12.498262 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c38207-5301-4a1a-a146-a9dc0890154a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:12.499890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c38207-5301-4a1a-a146-a9dc0890154a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:12.513278 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1256/2557] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:12.517636 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9e343-5014-4e74-8fd9-4ba003fce96c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:12.529089 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9e343-5014-4e74-8fd9-4ba003fce96c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:12.534423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daaafac-06ba-49e3-a30a-99e74b9d8fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:12.535103 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daaafac-06ba-49e3-a30a-99e74b9d8fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:12.535406 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daaafac-06ba-49e3-a30a-99e74b9d8fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:12.535670 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daaafac-06ba-49e3-a30a-99e74b9d8fb1 admin admin] PATH_INFO: `/v3/projects/2f4a30ddfcb04c569f1789cd5094ab94/users/5133a0fa6e2249b79212e804a2337a80/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:12.553404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daaafac-06ba-49e3-a30a-99e74b9d8fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f4a30ddfcb04c569f1789cd5094ab94, user_id=5133a0fa6e2249b79212e804a2337a80, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:12.556118 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daaafac-06ba-49e3-a30a-99e74b9d8fb1 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:12.566845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:12.572715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1302/2558] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:12 2022] PUT /identity/v3/projects/2f4a30ddfcb04c569f1789cd5094ab94/users/5133a0fa6e2249b79212e804a2337a80/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:12.585237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6532b-1c66-4680-a260-9f43e5ff6707 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:12.585537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6532b-1c66-4680-a260-9f43e5ff6707 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:12.585886 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6532b-1c66-4680-a260-9f43e5ff6707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:12.617663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-dff6532b-1c66-4680-a260-9f43e5ff6707 None None] MFA Rules not processed for user `5133a0fa6e2249b79212e804a2337a80`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:12.618887 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-dff6532b-1c66-4680-a260-9f43e5ff6707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:12.693905 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1257/2559] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:12.699128 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daaafac-06ba-49e3-a30a-99e74b9d8fb1 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:12.791838 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daaafac-06ba-49e3-a30a-99e74b9d8fb1 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:12.803775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a269c-1d06-4b50-a728-40d2b120a6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:12.804506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a269c-1d06-4b50-a728-40d2b120a6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:12.804796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a269c-1d06-4b50-a728-40d2b120a6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:12.805073 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a269c-1d06-4b50-a728-40d2b120a6a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:12.805992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a269c-1d06-4b50-a728-40d2b120a6a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:12.808269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368a269c-1d06-4b50-a728-40d2b120a6a3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:12.860903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1303/2560] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:12 2022] POST /identity/v3/projects => generated 365 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:12.865463 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6532b-1c66-4680-a260-9f43e5ff6707 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:12.887243 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6532b-1c66-4680-a260-9f43e5ff6707 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:12.967374 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c2660-6f94-442c-871e-25b9cb11166e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:12.968286 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c2660-6f94-442c-871e-25b9cb11166e admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:12.968569 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c2660-6f94-442c-871e-25b9cb11166e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:12.968846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c2660-6f94-442c-871e-25b9cb11166e admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:12.969479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c2660-6f94-442c-871e-25b9cb11166e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:12.971456 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c2660-6f94-442c-871e-25b9cb11166e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:12.998875 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1258/2561] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:12 2022] POST /identity/v3/users => generated 289 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:39:13.000425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a269c-1d06-4b50-a728-40d2b120a6a3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:13.013005 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a269c-1d06-4b50-a728-40d2b120a6a3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:13.018372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe6def-9062-4eeb-97db-3419e8822735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:13.018939 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe6def-9062-4eeb-97db-3419e8822735 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:13.019260 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe6def-9062-4eeb-97db-3419e8822735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:13.019545 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe6def-9062-4eeb-97db-3419e8822735 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:13.020293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe6def-9062-4eeb-97db-3419e8822735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:13.022041 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe6def-9062-4eeb-97db-3419e8822735 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:13.034986 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1304/2562] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:13.039291 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c2660-6f94-442c-871e-25b9cb11166e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:13.044986 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:13.050731 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c2660-6f94-442c-871e-25b9cb11166e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:13.056634 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bffb5e-1c23-4ee9-b29b-ccbd95a26c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:13.058780 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bffb5e-1c23-4ee9-b29b-ccbd95a26c23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:13.058908 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bffb5e-1c23-4ee9-b29b-ccbd95a26c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:13.059292 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bffb5e-1c23-4ee9-b29b-ccbd95a26c23 admin admin] PATH_INFO: `/v3/projects/01bc3b56cc7b4d67ae018c898405ca86/users/8ef360d028b2402da74d92348089b7b6/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:13.076040 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bffb5e-1c23-4ee9-b29b-ccbd95a26c23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01bc3b56cc7b4d67ae018c898405ca86, user_id=8ef360d028b2402da74d92348089b7b6, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:13.079000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bffb5e-1c23-4ee9-b29b-ccbd95a26c23 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:13.093920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1259/2563] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:13 2022] PUT /identity/v3/projects/01bc3b56cc7b4d67ae018c898405ca86/users/8ef360d028b2402da74d92348089b7b6/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:13.097899 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe6def-9062-4eeb-97db-3419e8822735 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:13.109092 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe6def-9062-4eeb-97db-3419e8822735 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:13.169032 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df963935-e7da-4440-a007-37594fa22c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:13.169657 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df963935-e7da-4440-a007-37594fa22c81 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:13.169936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df963935-e7da-4440-a007-37594fa22c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:13.170216 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df963935-e7da-4440-a007-37594fa22c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:13.170865 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df963935-e7da-4440-a007-37594fa22c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:13.172510 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df963935-e7da-4440-a007-37594fa22c81 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:13.184062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1305/2564] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:13.188509 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bffb5e-1c23-4ee9-b29b-ccbd95a26c23 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:13.199435 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bffb5e-1c23-4ee9-b29b-ccbd95a26c23 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:13.204359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a37ca0-71b5-452f-a52e-7c704626201e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:13.205008 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a37ca0-71b5-452f-a52e-7c704626201e admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:13.205281 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a37ca0-71b5-452f-a52e-7c704626201e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:13.205550 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a37ca0-71b5-452f-a52e-7c704626201e admin admin] PATH_INFO: `/v3/projects/01bc3b56cc7b4d67ae018c898405ca86/users/8ef360d028b2402da74d92348089b7b6/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:13.207481 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a37ca0-71b5-452f-a52e-7c704626201e admin admin] RBAC: Authorizing `identity:create_grant(project_id=01bc3b56cc7b4d67ae018c898405ca86, user_id=8ef360d028b2402da74d92348089b7b6, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:13.210205 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a37ca0-71b5-452f-a52e-7c704626201e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:13.220457 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1260/2565] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:13 2022] PUT /identity/v3/projects/01bc3b56cc7b4d67ae018c898405ca86/users/8ef360d028b2402da74d92348089b7b6/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:13.230603 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2cf72-a186-4f47-b571-b6f0f2652f35 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:13.230904 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2cf72-a186-4f47-b571-b6f0f2652f35 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:13.231184 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2cf72-a186-4f47-b571-b6f0f2652f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:13.262407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b2a2cf72-a186-4f47-b571-b6f0f2652f35 None None] MFA Rules not processed for user `8ef360d028b2402da74d92348089b7b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:13.263655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b2a2cf72-a186-4f47-b571-b6f0f2652f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:13.329873 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:13.343089 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1306/2566] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:13.356996 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a37ca0-71b5-452f-a52e-7c704626201e admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:13.358821 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-58a37ca0-71b5-452f-a52e-7c704626201e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:13.421718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a37ca0-71b5-452f-a52e-7c704626201e admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:13.425475 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98028a-faab-4cbb-91ac-4ec0d41466e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:13.426074 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98028a-faab-4cbb-91ac-4ec0d41466e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:13.426351 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98028a-faab-4cbb-91ac-4ec0d41466e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:13.426625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98028a-faab-4cbb-91ac-4ec0d41466e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:13.440198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98028a-faab-4cbb-91ac-4ec0d41466e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:13.441660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98028a-faab-4cbb-91ac-4ec0d41466e5 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:13.455819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1261/2567] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:13 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:13.543231 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2cf72-a186-4f47-b571-b6f0f2652f35 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:13.554827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2cf72-a186-4f47-b571-b6f0f2652f35 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:13.560512 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43baa0-928d-45dd-a09e-d74750085a64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:13.561285 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43baa0-928d-45dd-a09e-d74750085a64 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:13.561751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43baa0-928d-45dd-a09e-d74750085a64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:13.563149 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43baa0-928d-45dd-a09e-d74750085a64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:13.643975 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43baa0-928d-45dd-a09e-d74750085a64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:13.645447 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43baa0-928d-45dd-a09e-d74750085a64 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:13.655819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1307/2568] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:23.450251 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=40,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:23.651439 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=30 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:24.086129 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98028a-faab-4cbb-91ac-4ec0d41466e5 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:24.151028 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98028a-faab-4cbb-91ac-4ec0d41466e5 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:24.155498 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48443f0c-e9b2-433d-90c0-34aa1190c910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:24.156166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48443f0c-e9b2-433d-90c0-34aa1190c910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:24.156444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48443f0c-e9b2-433d-90c0-34aa1190c910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:24.156724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48443f0c-e9b2-433d-90c0-34aa1190c910 admin admin] PATH_INFO: `/v3/users/5133a0fa6e2249b79212e804a2337a80` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:24.157865 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48443f0c-e9b2-433d-90c0-34aa1190c910 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5133a0fa6e2249b79212e804a2337a80)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:24.159770 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48443f0c-e9b2-433d-90c0-34aa1190c910 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:24.224381 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-48443f0c-e9b2-433d-90c0-34aa1190c910 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5133a0fa6e2249b79212e804a2337a80'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:24.233358 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-48443f0c-e9b2-433d-90c0-34aa1190c910 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5133a0fa6e2249b79212e804a2337a80'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:24.246550 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-48443f0c-e9b2-433d-90c0-34aa1190c910 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5133a0fa6e2249b79212e804a2337a80'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:24.247704 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-48443f0c-e9b2-433d-90c0-34aa1190c910 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5133a0fa6e2249b79212e804a2337a80'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:24.264834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-48443f0c-e9b2-433d-90c0-34aa1190c910 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5133a0fa6e2249b79212e804a2337a80'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:24.269596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1262/2569] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:24 2022] DELETE /identity/v3/users/5133a0fa6e2249b79212e804a2337a80 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:24.274354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924adae-2d50-4423-b428-9c786667cac4 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:24.274639 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924adae-2d50-4423-b428-9c786667cac4 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:24.274915 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924adae-2d50-4423-b428-9c786667cac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:24.300270 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d924adae-2d50-4423-b428-9c786667cac4 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:24.306870 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d924adae-2d50-4423-b428-9c786667cac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:24.378664 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1308/2570] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:24.391562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48443f0c-e9b2-433d-90c0-34aa1190c910 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:24.391562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-48443f0c-e9b2-433d-90c0-34aa1190c910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:24.454985 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48443f0c-e9b2-433d-90c0-34aa1190c910 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:24.458782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48331218-e70f-42e6-bb2a-95fd0d18fb6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:24.459387 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48331218-e70f-42e6-bb2a-95fd0d18fb6e service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:24.459663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48331218-e70f-42e6-bb2a-95fd0d18fb6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:24.459956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48331218-e70f-42e6-bb2a-95fd0d18fb6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:24.471223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48331218-e70f-42e6-bb2a-95fd0d18fb6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:24.472769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48331218-e70f-42e6-bb2a-95fd0d18fb6e service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:24.483618 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1263/2571] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:24.771240 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d924adae-2d50-4423-b428-9c786667cac4 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:24.772735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d924adae-2d50-4423-b428-9c786667cac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:24.801591 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d924adae-2d50-4423-b428-9c786667cac4 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:24.807298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086eca65-e76a-4d48-9876-cd20f16c6ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:24.808500 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086eca65-e76a-4d48-9876-cd20f16c6ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:24.809028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086eca65-e76a-4d48-9876-cd20f16c6ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:24.809514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086eca65-e76a-4d48-9876-cd20f16c6ad2 admin admin] PATH_INFO: `/v3/projects/2f4a30ddfcb04c569f1789cd5094ab94` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:24.811691 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086eca65-e76a-4d48-9876-cd20f16c6ad2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f4a30ddfcb04c569f1789cd5094ab94)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:24.814541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086eca65-e76a-4d48-9876-cd20f16c6ad2 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:24.876384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-086eca65-e76a-4d48-9876-cd20f16c6ad2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f4a30ddfcb04c569f1789cd5094ab94', 'request_id': 'req-086eca65-e76a-4d48-9876-cd20f16c6ad2'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:24.886548 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-086eca65-e76a-4d48-9876-cd20f16c6ad2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f4a30ddfcb04c569f1789cd5094ab94', 'request_id': 'req-086eca65-e76a-4d48-9876-cd20f16c6ad2'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:24.896473 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-086eca65-e76a-4d48-9876-cd20f16c6ad2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f4a30ddfcb04c569f1789cd5094ab94', 'request_id': 'req-086eca65-e76a-4d48-9876-cd20f16c6ad2'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:24.910984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-086eca65-e76a-4d48-9876-cd20f16c6ad2 admin admin] The token cache is being invalidate because project 2f4a30ddfcb04c569f1789cd5094ab94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:39:24.911324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-086eca65-e76a-4d48-9876-cd20f16c6ad2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:24.912090 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-086eca65-e76a-4d48-9876-cd20f16c6ad2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:24.918548 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1309/2572] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:24 2022] DELETE /identity/v3/projects/2f4a30ddfcb04c569f1789cd5094ab94 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:24.919994 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48331218-e70f-42e6-bb2a-95fd0d18fb6e service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:24.940807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48331218-e70f-42e6-bb2a-95fd0d18fb6e service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:24.972503 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:25.023602 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b606d7-3d1b-4ae9-8463-104f8d2d9207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:25.024545 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b606d7-3d1b-4ae9-8463-104f8d2d9207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:25.024829 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b606d7-3d1b-4ae9-8463-104f8d2d9207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:25.025104 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b606d7-3d1b-4ae9-8463-104f8d2d9207 admin admin] PATH_INFO: `/v3/users/8ef360d028b2402da74d92348089b7b6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:25.026402 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b606d7-3d1b-4ae9-8463-104f8d2d9207 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ef360d028b2402da74d92348089b7b6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:25.028371 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b606d7-3d1b-4ae9-8463-104f8d2d9207 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:25.092096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-60b606d7-3d1b-4ae9-8463-104f8d2d9207 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ef360d028b2402da74d92348089b7b6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:25.099781 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-60b606d7-3d1b-4ae9-8463-104f8d2d9207 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ef360d028b2402da74d92348089b7b6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:25.108508 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-60b606d7-3d1b-4ae9-8463-104f8d2d9207 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ef360d028b2402da74d92348089b7b6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:25.109136 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-60b606d7-3d1b-4ae9-8463-104f8d2d9207 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ef360d028b2402da74d92348089b7b6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:25.123485 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-60b606d7-3d1b-4ae9-8463-104f8d2d9207 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ef360d028b2402da74d92348089b7b6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:25.130521 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1264/2573] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:24 2022] DELETE /identity/v3/users/8ef360d028b2402da74d92348089b7b6 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:25.441199 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086eca65-e76a-4d48-9876-cd20f16c6ad2 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:25.442936 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-086eca65-e76a-4d48-9876-cd20f16c6ad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:25.464971 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086eca65-e76a-4d48-9876-cd20f16c6ad2 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:25.548821 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0fae5-384d-45c6-aff2-ae9484ae3fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:25.549465 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0fae5-384d-45c6-aff2-ae9484ae3fa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:25.549745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0fae5-384d-45c6-aff2-ae9484ae3fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:25.550049 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0fae5-384d-45c6-aff2-ae9484ae3fa0 admin admin] PATH_INFO: `/v3/projects/01bc3b56cc7b4d67ae018c898405ca86` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:25.551188 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0fae5-384d-45c6-aff2-ae9484ae3fa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01bc3b56cc7b4d67ae018c898405ca86)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:25.553166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0fae5-384d-45c6-aff2-ae9484ae3fa0 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:25.590718 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:25.616312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-79a0fae5-384d-45c6-aff2-ae9484ae3fa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01bc3b56cc7b4d67ae018c898405ca86', 'request_id': 'req-79a0fae5-384d-45c6-aff2-ae9484ae3fa0'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:25.631445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-79a0fae5-384d-45c6-aff2-ae9484ae3fa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01bc3b56cc7b4d67ae018c898405ca86', 'request_id': 'req-79a0fae5-384d-45c6-aff2-ae9484ae3fa0'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:25.640128 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-79a0fae5-384d-45c6-aff2-ae9484ae3fa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01bc3b56cc7b4d67ae018c898405ca86', 'request_id': 'req-79a0fae5-384d-45c6-aff2-ae9484ae3fa0'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:25.654305 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-79a0fae5-384d-45c6-aff2-ae9484ae3fa0 admin admin] The token cache is being invalidate because project 01bc3b56cc7b4d67ae018c898405ca86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:39:25.654735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-79a0fae5-384d-45c6-aff2-ae9484ae3fa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:25.655486 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-79a0fae5-384d-45c6-aff2-ae9484ae3fa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:25.659984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1310/2574] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:25 2022] DELETE /identity/v3/projects/01bc3b56cc7b4d67ae018c898405ca86 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:25.662474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb9719-b9a6-48e1-950f-02c26196aa5d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:25.662974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb9719-b9a6-48e1-950f-02c26196aa5d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:25.663078 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb9719-b9a6-48e1-950f-02c26196aa5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:25.692679 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-08bb9719-b9a6-48e1-950f-02c26196aa5d None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:25.693905 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-08bb9719-b9a6-48e1-950f-02c26196aa5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:25.761446 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1265/2575] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:25.769556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a269ec9-752e-4f4f-92cf-d1430fd11a50 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:25.769849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a269ec9-752e-4f4f-92cf-d1430fd11a50 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:25.770129 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a269ec9-752e-4f4f-92cf-d1430fd11a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:25.792459 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-6a269ec9-752e-4f4f-92cf-d1430fd11a50 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:25.793723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6a269ec9-752e-4f4f-92cf-d1430fd11a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:25.801812 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1311/2576] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:25.807043 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb9719-b9a6-48e1-950f-02c26196aa5d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:25.817678 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb9719-b9a6-48e1-950f-02c26196aa5d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:25.822752 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad65b9-a10d-484d-8aaf-9f18c3a79019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:25.823375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad65b9-a10d-484d-8aaf-9f18c3a79019 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:25.823696 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad65b9-a10d-484d-8aaf-9f18c3a79019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:25.824158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad65b9-a10d-484d-8aaf-9f18c3a79019 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:25.824653 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad65b9-a10d-484d-8aaf-9f18c3a79019 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:39:25.824997 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad65b9-a10d-484d-8aaf-9f18c3a79019 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:25.826759 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad65b9-a10d-484d-8aaf-9f18c3a79019 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:25.839328 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1266/2577] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:39:25.844360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a269ec9-752e-4f4f-92cf-d1430fd11a50 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:25.855376 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a269ec9-752e-4f4f-92cf-d1430fd11a50 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:25.860482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f5973-51ca-44ff-acf1-1c70afdb3fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:25.861130 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f5973-51ca-44ff-acf1-1c70afdb3fde admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:25.861406 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f5973-51ca-44ff-acf1-1c70afdb3fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:25.861687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f5973-51ca-44ff-acf1-1c70afdb3fde admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:25.862321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f5973-51ca-44ff-acf1-1c70afdb3fde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:25.864239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f5973-51ca-44ff-acf1-1c70afdb3fde admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:25.896019 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1312/2578] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:25 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:39:25.898102 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad65b9-a10d-484d-8aaf-9f18c3a79019 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:25.909219 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad65b9-a10d-484d-8aaf-9f18c3a79019 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:25.947919 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:25.991706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12327a-9b35-4dbf-87f6-10261032075c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:25.992622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12327a-9b35-4dbf-87f6-10261032075c admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:25.993088 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12327a-9b35-4dbf-87f6-10261032075c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:25.993507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12327a-9b35-4dbf-87f6-10261032075c admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:25.994523 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12327a-9b35-4dbf-87f6-10261032075c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:25.997688 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12327a-9b35-4dbf-87f6-10261032075c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:26.030128 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1267/2579] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:25 2022] POST /identity/v3/users => generated 284 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:39:26.034153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f5973-51ca-44ff-acf1-1c70afdb3fde admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:26.046267 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f5973-51ca-44ff-acf1-1c70afdb3fde admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:26.051412 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476df00a-c689-4433-a95f-95b04471fcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:26.052178 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476df00a-c689-4433-a95f-95b04471fcff admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:26.052541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476df00a-c689-4433-a95f-95b04471fcff admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:26.052916 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476df00a-c689-4433-a95f-95b04471fcff admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:26.053716 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476df00a-c689-4433-a95f-95b04471fcff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:26.056004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476df00a-c689-4433-a95f-95b04471fcff admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:26.071586 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1313/2580] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:26.075880 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12327a-9b35-4dbf-87f6-10261032075c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:26.094999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12327a-9b35-4dbf-87f6-10261032075c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:26.105585 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c82ae-1225-49cb-b3a5-9b509cd4c668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:26.106669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c82ae-1225-49cb-b3a5-9b509cd4c668 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:26.107331 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c82ae-1225-49cb-b3a5-9b509cd4c668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:26.107794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c82ae-1225-49cb-b3a5-9b509cd4c668 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:26.109108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c82ae-1225-49cb-b3a5-9b509cd4c668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:26.112077 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c82ae-1225-49cb-b3a5-9b509cd4c668 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:26.140200 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1268/2581] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:26.144511 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476df00a-c689-4433-a95f-95b04471fcff admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:26.164865 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476df00a-c689-4433-a95f-95b04471fcff admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:26.177602 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073d5b4-1439-4e4e-b91d-4bbccd3ad9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:26.178287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073d5b4-1439-4e4e-b91d-4bbccd3ad9a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:26.178591 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073d5b4-1439-4e4e-b91d-4bbccd3ad9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:26.178908 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073d5b4-1439-4e4e-b91d-4bbccd3ad9a3 admin admin] PATH_INFO: `/v3/projects/8e0eacbfb0654eeaae28aa781494f997/users/1a1c46cecf044ba5adfd52bdfd4de0f7/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:26.196652 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073d5b4-1439-4e4e-b91d-4bbccd3ad9a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e0eacbfb0654eeaae28aa781494f997, user_id=1a1c46cecf044ba5adfd52bdfd4de0f7, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:26.199488 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073d5b4-1439-4e4e-b91d-4bbccd3ad9a3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:26.214736 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1314/2582] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:26 2022] PUT /identity/v3/projects/8e0eacbfb0654eeaae28aa781494f997/users/1a1c46cecf044ba5adfd52bdfd4de0f7/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:39:26.226706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf425b2-a8f2-4cdb-bff0-af649409b271 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:26.227181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf425b2-a8f2-4cdb-bff0-af649409b271 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:26.228161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf425b2-a8f2-4cdb-bff0-af649409b271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:26.260197 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5cf425b2-a8f2-4cdb-bff0-af649409b271 None None] MFA Rules not processed for user `1a1c46cecf044ba5adfd52bdfd4de0f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:26.262064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5cf425b2-a8f2-4cdb-bff0-af649409b271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:26.356507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1269/2583] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:26.360405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073d5b4-1439-4e4e-b91d-4bbccd3ad9a3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:26.429340 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073d5b4-1439-4e4e-b91d-4bbccd3ad9a3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:26.434240 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f205a91-5d3a-46b9-8f75-78713184a4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:26.435130 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f205a91-5d3a-46b9-8f75-78713184a4cc admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:26.435416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f205a91-5d3a-46b9-8f75-78713184a4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:26.435696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f205a91-5d3a-46b9-8f75-78713184a4cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:26.436394 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f205a91-5d3a-46b9-8f75-78713184a4cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:26.438300 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f205a91-5d3a-46b9-8f75-78713184a4cc admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:26.472060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1315/2584] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:26 2022] POST /identity/v3/projects => generated 369 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:39:26.476375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf425b2-a8f2-4cdb-bff0-af649409b271 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:26.487637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf425b2-a8f2-4cdb-bff0-af649409b271 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:26.551258 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790dbef-f083-4a74-b189-cedf515ca10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:26.552151 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790dbef-f083-4a74-b189-cedf515ca10d admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:26.552607 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790dbef-f083-4a74-b189-cedf515ca10d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:26.555143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790dbef-f083-4a74-b189-cedf515ca10d admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:26.556163 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790dbef-f083-4a74-b189-cedf515ca10d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:26.558578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790dbef-f083-4a74-b189-cedf515ca10d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:26.583129 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1270/2585] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:26 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:39:26.590081 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f205a91-5d3a-46b9-8f75-78713184a4cc admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:26.609045 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f205a91-5d3a-46b9-8f75-78713184a4cc admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:26.616704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6236f2e-4e90-4e73-bff6-ec95a118de46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:26.617775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6236f2e-4e90-4e73-bff6-ec95a118de46 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:26.618982 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6236f2e-4e90-4e73-bff6-ec95a118de46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:26.619511 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6236f2e-4e90-4e73-bff6-ec95a118de46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:26.620726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6236f2e-4e90-4e73-bff6-ec95a118de46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:26.623667 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6236f2e-4e90-4e73-bff6-ec95a118de46 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:26.644932 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1316/2586] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:39:26.652097 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790dbef-f083-4a74-b189-cedf515ca10d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:26.670601 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790dbef-f083-4a74-b189-cedf515ca10d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:26.679036 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e071e-15a1-4040-8957-4b7b04fb886d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:26.680899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e071e-15a1-4040-8957-4b7b04fb886d admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:26.681495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e071e-15a1-4040-8957-4b7b04fb886d admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:26.682093 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e071e-15a1-4040-8957-4b7b04fb886d admin admin] PATH_INFO: `/v3/projects/8c21675704764786a84400625d19d725/users/d2ac8635e6a94dcb98135fbdf776a2c2/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:26.710986 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e071e-15a1-4040-8957-4b7b04fb886d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c21675704764786a84400625d19d725, user_id=d2ac8635e6a94dcb98135fbdf776a2c2, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:26.716654 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e071e-15a1-4040-8957-4b7b04fb886d admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:26.742148 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1271/2587] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:26 2022] PUT /identity/v3/projects/8c21675704764786a84400625d19d725/users/d2ac8635e6a94dcb98135fbdf776a2c2/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:26.748532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6236f2e-4e90-4e73-bff6-ec95a118de46 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:26.766872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6236f2e-4e90-4e73-bff6-ec95a118de46 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:26.792793 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:26.844859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29cafa-685f-4ea0-9493-f7628cf6f213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:26.845524 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29cafa-685f-4ea0-9493-f7628cf6f213 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:26.845811 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29cafa-685f-4ea0-9493-f7628cf6f213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:26.846088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29cafa-685f-4ea0-9493-f7628cf6f213 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:26.846776 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29cafa-685f-4ea0-9493-f7628cf6f213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:26.848426 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29cafa-685f-4ea0-9493-f7628cf6f213 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:26.859289 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1317/2588] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:26.863464 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e071e-15a1-4040-8957-4b7b04fb886d admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:26.871230 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:26.878130 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e071e-15a1-4040-8957-4b7b04fb886d admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:26.887706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921be26-2a98-4528-9cd2-fc68c9778d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:26.888555 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921be26-2a98-4528-9cd2-fc68c9778d32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:26.892394 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921be26-2a98-4528-9cd2-fc68c9778d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:26.892777 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921be26-2a98-4528-9cd2-fc68c9778d32 admin admin] PATH_INFO: `/v3/projects/8c21675704764786a84400625d19d725/users/d2ac8635e6a94dcb98135fbdf776a2c2/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:26.895694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921be26-2a98-4528-9cd2-fc68c9778d32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c21675704764786a84400625d19d725, user_id=d2ac8635e6a94dcb98135fbdf776a2c2, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:26.898486 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921be26-2a98-4528-9cd2-fc68c9778d32 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:26.912521 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1272/2589] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:26 2022] PUT /identity/v3/projects/8c21675704764786a84400625d19d725/users/d2ac8635e6a94dcb98135fbdf776a2c2/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:39:26.924355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6654b8-b6a4-44f9-85a5-9e6e4b621b12 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:26.924835 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6654b8-b6a4-44f9-85a5-9e6e4b621b12 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:26.925815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6654b8-b6a4-44f9-85a5-9e6e4b621b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:26.961258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-7c6654b8-b6a4-44f9-85a5-9e6e4b621b12 None None] MFA Rules not processed for user `d2ac8635e6a94dcb98135fbdf776a2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:26.963246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7c6654b8-b6a4-44f9-85a5-9e6e4b621b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:27.045824 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1318/2590] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:27.060680 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921be26-2a98-4528-9cd2-fc68c9778d32 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:27.063324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7921be26-2a98-4528-9cd2-fc68c9778d32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:27.124039 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921be26-2a98-4528-9cd2-fc68c9778d32 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:27.127709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51152f-708e-4f9c-ae62-98227a12323f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:27.128424 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51152f-708e-4f9c-ae62-98227a12323f service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:27.128709 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51152f-708e-4f9c-ae62-98227a12323f service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:27.128983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51152f-708e-4f9c-ae62-98227a12323f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:27.138734 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51152f-708e-4f9c-ae62-98227a12323f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:27.140182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51152f-708e-4f9c-ae62-98227a12323f service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:27.150504 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1273/2591] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:29.253395 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6654b8-b6a4-44f9-85a5-9e6e4b621b12 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:29.334975 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6654b8-b6a4-44f9-85a5-9e6e4b621b12 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:29.339741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01736aa4-65f9-4eb1-be24-ed0df894093d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:29.340431 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01736aa4-65f9-4eb1-be24-ed0df894093d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:29.340720 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01736aa4-65f9-4eb1-be24-ed0df894093d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:29.341000 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01736aa4-65f9-4eb1-be24-ed0df894093d admin admin] PATH_INFO: `/v3/users/1a1c46cecf044ba5adfd52bdfd4de0f7` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:29.342189 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01736aa4-65f9-4eb1-be24-ed0df894093d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a1c46cecf044ba5adfd52bdfd4de0f7)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:29.344157 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01736aa4-65f9-4eb1-be24-ed0df894093d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:29.417769 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-01736aa4-65f9-4eb1-be24-ed0df894093d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a1c46cecf044ba5adfd52bdfd4de0f7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:29.419399 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:29.432133 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-01736aa4-65f9-4eb1-be24-ed0df894093d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a1c46cecf044ba5adfd52bdfd4de0f7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:29.443698 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-01736aa4-65f9-4eb1-be24-ed0df894093d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a1c46cecf044ba5adfd52bdfd4de0f7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:29.444552 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-01736aa4-65f9-4eb1-be24-ed0df894093d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a1c46cecf044ba5adfd52bdfd4de0f7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:29.445254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-01736aa4-65f9-4eb1-be24-ed0df894093d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a1c46cecf044ba5adfd52bdfd4de0f7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:29.469346 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1319/2592] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:29 2022] DELETE /identity/v3/users/1a1c46cecf044ba5adfd52bdfd4de0f7 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:29.474299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bfe72-356b-4011-a707-fddb7d24329a None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:29.475082 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bfe72-356b-4011-a707-fddb7d24329a None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:29.475082 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bfe72-356b-4011-a707-fddb7d24329a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:29.498265 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b72bfe72-356b-4011-a707-fddb7d24329a None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:29.499926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b72bfe72-356b-4011-a707-fddb7d24329a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:29.578858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1274/2593] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:39:29.593048 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01736aa4-65f9-4eb1-be24-ed0df894093d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:29.595169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-01736aa4-65f9-4eb1-be24-ed0df894093d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:29.685314 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01736aa4-65f9-4eb1-be24-ed0df894093d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:29.691447 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4932a96-2ce0-406c-bb3c-d15ae47a1ed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:29.692541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4932a96-2ce0-406c-bb3c-d15ae47a1ed3 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:29.693048 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4932a96-2ce0-406c-bb3c-d15ae47a1ed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:29.693620 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4932a96-2ce0-406c-bb3c-d15ae47a1ed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:29.705784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4932a96-2ce0-406c-bb3c-d15ae47a1ed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:29.707520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4932a96-2ce0-406c-bb3c-d15ae47a1ed3 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:29.720447 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1320/2594] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:30.073126 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bfe72-356b-4011-a707-fddb7d24329a None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:30.074717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b72bfe72-356b-4011-a707-fddb7d24329a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:30.088843 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bfe72-356b-4011-a707-fddb7d24329a None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:30.094229 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb3ed3-a5f4-44a1-9dc1-24346ca03fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:30.094887 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb3ed3-a5f4-44a1-9dc1-24346ca03fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:30.095161 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb3ed3-a5f4-44a1-9dc1-24346ca03fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:30.095428 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb3ed3-a5f4-44a1-9dc1-24346ca03fad admin admin] PATH_INFO: `/v3/projects/8e0eacbfb0654eeaae28aa781494f997` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:30.096600 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb3ed3-a5f4-44a1-9dc1-24346ca03fad admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e0eacbfb0654eeaae28aa781494f997)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:30.098506 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb3ed3-a5f4-44a1-9dc1-24346ca03fad admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:30.164471 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-36eb3ed3-a5f4-44a1-9dc1-24346ca03fad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e0eacbfb0654eeaae28aa781494f997', 'request_id': 'req-36eb3ed3-a5f4-44a1-9dc1-24346ca03fad'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:30.175655 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-36eb3ed3-a5f4-44a1-9dc1-24346ca03fad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e0eacbfb0654eeaae28aa781494f997', 'request_id': 'req-36eb3ed3-a5f4-44a1-9dc1-24346ca03fad'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:30.188848 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-36eb3ed3-a5f4-44a1-9dc1-24346ca03fad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e0eacbfb0654eeaae28aa781494f997', 'request_id': 'req-36eb3ed3-a5f4-44a1-9dc1-24346ca03fad'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:30.208657 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-36eb3ed3-a5f4-44a1-9dc1-24346ca03fad admin admin] The token cache is being invalidate because project 8e0eacbfb0654eeaae28aa781494f997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:39:30.208976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-36eb3ed3-a5f4-44a1-9dc1-24346ca03fad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:30.209613 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-36eb3ed3-a5f4-44a1-9dc1-24346ca03fad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:30.211150 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1275/2595] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:30 2022] DELETE /identity/v3/projects/8e0eacbfb0654eeaae28aa781494f997 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:30.215262 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4932a96-2ce0-406c-bb3c-d15ae47a1ed3 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:30.225876 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4932a96-2ce0-406c-bb3c-d15ae47a1ed3 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:30.349253 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b7413-77f4-430e-a32f-e80ac7f0bbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:30.351172 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b7413-77f4-430e-a32f-e80ac7f0bbd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:30.351682 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b7413-77f4-430e-a32f-e80ac7f0bbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:30.352297 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b7413-77f4-430e-a32f-e80ac7f0bbd7 admin admin] PATH_INFO: `/v3/users/d2ac8635e6a94dcb98135fbdf776a2c2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:30.354182 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b7413-77f4-430e-a32f-e80ac7f0bbd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2ac8635e6a94dcb98135fbdf776a2c2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:30.357410 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b7413-77f4-430e-a32f-e80ac7f0bbd7 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:30.438380 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4b0b7413-77f4-430e-a32f-e80ac7f0bbd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2ac8635e6a94dcb98135fbdf776a2c2'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:30.448663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4b0b7413-77f4-430e-a32f-e80ac7f0bbd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2ac8635e6a94dcb98135fbdf776a2c2'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:30.457662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4b0b7413-77f4-430e-a32f-e80ac7f0bbd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2ac8635e6a94dcb98135fbdf776a2c2'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:30.458468 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4b0b7413-77f4-430e-a32f-e80ac7f0bbd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2ac8635e6a94dcb98135fbdf776a2c2'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:30.459237 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-4b0b7413-77f4-430e-a32f-e80ac7f0bbd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2ac8635e6a94dcb98135fbdf776a2c2'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:30.468471 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:30.498442 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1321/2596] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:30 2022] DELETE /identity/v3/users/d2ac8635e6a94dcb98135fbdf776a2c2 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:30.965515 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb3ed3-a5f4-44a1-9dc1-24346ca03fad admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:30.967831 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-36eb3ed3-a5f4-44a1-9dc1-24346ca03fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:30.982841 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb3ed3-a5f4-44a1-9dc1-24346ca03fad admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:30.999294 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:31.100637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87506d10-2fa6-4933-8c1d-f8b5471cb35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:31.101290 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87506d10-2fa6-4933-8c1d-f8b5471cb35e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:31.101565 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87506d10-2fa6-4933-8c1d-f8b5471cb35e admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:31.101837 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87506d10-2fa6-4933-8c1d-f8b5471cb35e admin admin] PATH_INFO: `/v3/projects/8c21675704764786a84400625d19d725` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:31.103132 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87506d10-2fa6-4933-8c1d-f8b5471cb35e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c21675704764786a84400625d19d725)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:31.105711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87506d10-2fa6-4933-8c1d-f8b5471cb35e admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:31.199765 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-87506d10-2fa6-4933-8c1d-f8b5471cb35e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c21675704764786a84400625d19d725', 'request_id': 'req-87506d10-2fa6-4933-8c1d-f8b5471cb35e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:31.212668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-87506d10-2fa6-4933-8c1d-f8b5471cb35e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c21675704764786a84400625d19d725', 'request_id': 'req-87506d10-2fa6-4933-8c1d-f8b5471cb35e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:31.226000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-87506d10-2fa6-4933-8c1d-f8b5471cb35e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c21675704764786a84400625d19d725', 'request_id': 'req-87506d10-2fa6-4933-8c1d-f8b5471cb35e'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:31.249264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-87506d10-2fa6-4933-8c1d-f8b5471cb35e admin admin] The token cache is being invalidate because project 8c21675704764786a84400625d19d725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:39:31.252635 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-87506d10-2fa6-4933-8c1d-f8b5471cb35e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:31.253442 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-87506d10-2fa6-4933-8c1d-f8b5471cb35e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:31.261625 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1276/2597] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:30 2022] DELETE /identity/v3/projects/8c21675704764786a84400625d19d725 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:31.266511 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b166b36-5a64-4b81-84b5-f23a073b23bf None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:31.267057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b166b36-5a64-4b81-84b5-f23a073b23bf None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:31.267508 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b166b36-5a64-4b81-84b5-f23a073b23bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:31.305327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2b166b36-5a64-4b81-84b5-f23a073b23bf None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:31.307459 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2b166b36-5a64-4b81-84b5-f23a073b23bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:31.468182 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1322/2598] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:31.472093 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab9e5c-24dc-44ad-a4b3-c28f1a45e22e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:31.472492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab9e5c-24dc-44ad-a4b3-c28f1a45e22e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:31.472939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab9e5c-24dc-44ad-a4b3-c28f1a45e22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:31.515694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-dbab9e5c-24dc-44ad-a4b3-c28f1a45e22e None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:31.517938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-dbab9e5c-24dc-44ad-a4b3-c28f1a45e22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:31.534137 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1277/2599] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:31.539490 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b166b36-5a64-4b81-84b5-f23a073b23bf None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:31.554482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b166b36-5a64-4b81-84b5-f23a073b23bf None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:31.562796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d215bd7-68cf-4e40-a846-74dceec98171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:31.563731 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d215bd7-68cf-4e40-a846-74dceec98171 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:31.564210 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d215bd7-68cf-4e40-a846-74dceec98171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:31.564628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d215bd7-68cf-4e40-a846-74dceec98171 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:31.565977 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d215bd7-68cf-4e40-a846-74dceec98171 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:39:31.566614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d215bd7-68cf-4e40-a846-74dceec98171 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:31.569465 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d215bd7-68cf-4e40-a846-74dceec98171 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:31.591493 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1323/2600] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:39:31.596596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab9e5c-24dc-44ad-a4b3-c28f1a45e22e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:31.608916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab9e5c-24dc-44ad-a4b3-c28f1a45e22e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:31.616859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f229a-b93a-484c-b589-3fcf011de3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:31.617554 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f229a-b93a-484c-b589-3fcf011de3b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:31.617970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f229a-b93a-484c-b589-3fcf011de3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:31.618243 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f229a-b93a-484c-b589-3fcf011de3b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:31.618892 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f229a-b93a-484c-b589-3fcf011de3b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:31.621792 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f229a-b93a-484c-b589-3fcf011de3b1 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:31.675645 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1278/2601] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:31 2022] POST /identity/v3/projects => generated 391 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:39:31.679796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d215bd7-68cf-4e40-a846-74dceec98171 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:31.699744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d215bd7-68cf-4e40-a846-74dceec98171 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:31.812506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f414a2-c809-467e-8500-ea9d6403fcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:31.813564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f414a2-c809-467e-8500-ea9d6403fcbe admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:31.814075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f414a2-c809-467e-8500-ea9d6403fcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:31.814582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f414a2-c809-467e-8500-ea9d6403fcbe admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:31.815897 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f414a2-c809-467e-8500-ea9d6403fcbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:31.819682 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f414a2-c809-467e-8500-ea9d6403fcbe admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:31.862437 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1324/2602] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:31 2022] POST /identity/v3/users => generated 296 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:39:31.864578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f229a-b93a-484c-b589-3fcf011de3b1 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:31.887220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f229a-b93a-484c-b589-3fcf011de3b1 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:31.893646 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8631fc-c681-4eb2-bc8c-52b10ad71376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:31.894730 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8631fc-c681-4eb2-bc8c-52b10ad71376 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:31.895203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8631fc-c681-4eb2-bc8c-52b10ad71376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:31.895717 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8631fc-c681-4eb2-bc8c-52b10ad71376 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:31.899609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8631fc-c681-4eb2-bc8c-52b10ad71376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:31.903984 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8631fc-c681-4eb2-bc8c-52b10ad71376 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:31.923231 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1279/2603] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:31.928265 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f414a2-c809-467e-8500-ea9d6403fcbe admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:31.951861 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f414a2-c809-467e-8500-ea9d6403fcbe admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:31.957617 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed2370-a0a3-4e29-951d-fae35a81efff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:31.958688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed2370-a0a3-4e29-951d-fae35a81efff admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:31.958981 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed2370-a0a3-4e29-951d-fae35a81efff admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:31.959254 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed2370-a0a3-4e29-951d-fae35a81efff admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:31.960158 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed2370-a0a3-4e29-951d-fae35a81efff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:31.964198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed2370-a0a3-4e29-951d-fae35a81efff admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:31.989099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1325/2604] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:31.995710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8631fc-c681-4eb2-bc8c-52b10ad71376 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:32.025925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8631fc-c681-4eb2-bc8c-52b10ad71376 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:32.034747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c8a2c-25d8-4a65-873a-ef8af67fef2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:32.036189 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c8a2c-25d8-4a65-873a-ef8af67fef2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:32.036841 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c8a2c-25d8-4a65-873a-ef8af67fef2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:32.037252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c8a2c-25d8-4a65-873a-ef8af67fef2b admin admin] PATH_INFO: `/v3/projects/8fa009d7e07e47aeb5fa9adbbdabda13/users/605cfc72dd614e7ebe12811a0e48c200/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:32.062992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c8a2c-25d8-4a65-873a-ef8af67fef2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa009d7e07e47aeb5fa9adbbdabda13, user_id=605cfc72dd614e7ebe12811a0e48c200, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:32.067768 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c8a2c-25d8-4a65-873a-ef8af67fef2b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:32.097542 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1280/2605] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:31 2022] PUT /identity/v3/projects/8fa009d7e07e47aeb5fa9adbbdabda13/users/605cfc72dd614e7ebe12811a0e48c200/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:32.114679 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a755608-2e18-4b18-8458-bf5fbd2a2f3b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:32.115165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a755608-2e18-4b18-8458-bf5fbd2a2f3b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:32.115618 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a755608-2e18-4b18-8458-bf5fbd2a2f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:32.200475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2a755608-2e18-4b18-8458-bf5fbd2a2f3b None None] MFA Rules not processed for user `605cfc72dd614e7ebe12811a0e48c200`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:32.202995 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2a755608-2e18-4b18-8458-bf5fbd2a2f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:32.293887 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:32.356462 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1326/2606] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:32.363001 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c8a2c-25d8-4a65-873a-ef8af67fef2b admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:32.450118 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:32.480931 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c8a2c-25d8-4a65-873a-ef8af67fef2b admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:32.487142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8be5f9-66fe-492f-97c4-af9d00592a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:32.487785 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8be5f9-66fe-492f-97c4-af9d00592a6b admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:32.488194 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8be5f9-66fe-492f-97c4-af9d00592a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:32.488379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8be5f9-66fe-492f-97c4-af9d00592a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:32.489131 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8be5f9-66fe-492f-97c4-af9d00592a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:32.491066 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8be5f9-66fe-492f-97c4-af9d00592a6b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:32.531982 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1281/2607] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:32 2022] POST /identity/v3/projects => generated 391 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:32.535306 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a755608-2e18-4b18-8458-bf5fbd2a2f3b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:32.557404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a755608-2e18-4b18-8458-bf5fbd2a2f3b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:32.683672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c009f30-8dd2-4a3d-ab7c-0e9d12fd7ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:32.684448 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c009f30-8dd2-4a3d-ab7c-0e9d12fd7ddc admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:32.684982 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c009f30-8dd2-4a3d-ab7c-0e9d12fd7ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:32.685252 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c009f30-8dd2-4a3d-ab7c-0e9d12fd7ddc admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:32.685902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c009f30-8dd2-4a3d-ab7c-0e9d12fd7ddc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:32.690853 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c009f30-8dd2-4a3d-ab7c-0e9d12fd7ddc admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:32.748060 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1327/2608] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:32 2022] POST /identity/v3/users => generated 302 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:32.754836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8be5f9-66fe-492f-97c4-af9d00592a6b admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:32.778721 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8be5f9-66fe-492f-97c4-af9d00592a6b admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:32.794876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42246e4-31a7-46b0-8fbf-9c05e588c1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:32.797376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42246e4-31a7-46b0-8fbf-9c05e588c1e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:32.797376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42246e4-31a7-46b0-8fbf-9c05e588c1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:32.797659 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42246e4-31a7-46b0-8fbf-9c05e588c1e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:32.798845 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42246e4-31a7-46b0-8fbf-9c05e588c1e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:32.802044 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42246e4-31a7-46b0-8fbf-9c05e588c1e5 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:32.825904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1282/2609] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:39:32.830978 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c009f30-8dd2-4a3d-ab7c-0e9d12fd7ddc admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:32.846815 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c009f30-8dd2-4a3d-ab7c-0e9d12fd7ddc admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:32.855017 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43436a1-272c-4027-8ec9-9e76528165d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:32.855447 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43436a1-272c-4027-8ec9-9e76528165d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:32.855809 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43436a1-272c-4027-8ec9-9e76528165d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:32.856045 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43436a1-272c-4027-8ec9-9e76528165d7 admin admin] PATH_INFO: `/v3/projects/068c287023984e35aceaf3f3f88acfae/users/13d3ac0a782643a18676807770f562ff/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:32.879249 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43436a1-272c-4027-8ec9-9e76528165d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=068c287023984e35aceaf3f3f88acfae, user_id=13d3ac0a782643a18676807770f562ff, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:32.884730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43436a1-272c-4027-8ec9-9e76528165d7 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:32.907089 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1328/2610] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:32 2022] PUT /identity/v3/projects/068c287023984e35aceaf3f3f88acfae/users/13d3ac0a782643a18676807770f562ff/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:32.911774 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42246e4-31a7-46b0-8fbf-9c05e588c1e5 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:32.926164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42246e4-31a7-46b0-8fbf-9c05e588c1e5 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:33.011173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de74e4-46ef-45bd-be03-b6bfbf8e86c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:33.012196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de74e4-46ef-45bd-be03-b6bfbf8e86c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:33.012810 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de74e4-46ef-45bd-be03-b6bfbf8e86c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:33.013391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de74e4-46ef-45bd-be03-b6bfbf8e86c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:33.014533 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de74e4-46ef-45bd-be03-b6bfbf8e86c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:33.017538 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de74e4-46ef-45bd-be03-b6bfbf8e86c6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:33.034244 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1283/2611] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:39:33.042179 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43436a1-272c-4027-8ec9-9e76528165d7 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:33.077493 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43436a1-272c-4027-8ec9-9e76528165d7 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:33.086853 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effdf6aa-ea39-4220-b248-0faa47034e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:33.087852 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdf6aa-ea39-4220-b248-0faa47034e4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:33.088320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdf6aa-ea39-4220-b248-0faa47034e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:33.088736 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdf6aa-ea39-4220-b248-0faa47034e4e admin admin] PATH_INFO: `/v3/projects/068c287023984e35aceaf3f3f88acfae/users/13d3ac0a782643a18676807770f562ff/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:33.091456 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effdf6aa-ea39-4220-b248-0faa47034e4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=068c287023984e35aceaf3f3f88acfae, user_id=13d3ac0a782643a18676807770f562ff, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:33.094258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effdf6aa-ea39-4220-b248-0faa47034e4e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:33.106803 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1329/2612] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:33 2022] PUT /identity/v3/projects/068c287023984e35aceaf3f3f88acfae/users/13d3ac0a782643a18676807770f562ff/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:33.125250 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a810b76-c74f-4110-9325-888fa3982c05 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:33.125641 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a810b76-c74f-4110-9325-888fa3982c05 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:33.125976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a810b76-c74f-4110-9325-888fa3982c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:33.181201 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-4a810b76-c74f-4110-9325-888fa3982c05 None None] MFA Rules not processed for user `13d3ac0a782643a18676807770f562ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:33.182444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4a810b76-c74f-4110-9325-888fa3982c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:33.280723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1284/2613] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:33 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:39:33.292099 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effdf6aa-ea39-4220-b248-0faa47034e4e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:33.293877 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-effdf6aa-ea39-4220-b248-0faa47034e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:33.372454 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effdf6aa-ea39-4220-b248-0faa47034e4e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:33.379383 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a0631-d880-46c1-aa39-3f02a10b9cb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:33.380622 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a0631-d880-46c1-aa39-3f02a10b9cb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:33.381113 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a0631-d880-46c1-aa39-3f02a10b9cb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:33.381600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a0631-d880-46c1-aa39-3f02a10b9cb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:33.400285 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a0631-d880-46c1-aa39-3f02a10b9cb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:33.402982 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a0631-d880-46c1-aa39-3f02a10b9cb0 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:33.430777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1330/2614] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:33 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:33.513515 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a810b76-c74f-4110-9325-888fa3982c05 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:33.516048 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4a810b76-c74f-4110-9325-888fa3982c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:33.543613 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a810b76-c74f-4110-9325-888fa3982c05 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:33.552771 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ee496-cd63-49d7-822a-5404129b87f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:33.553732 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ee496-cd63-49d7-822a-5404129b87f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:33.554224 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ee496-cd63-49d7-822a-5404129b87f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:33.554904 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ee496-cd63-49d7-822a-5404129b87f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:33.609049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:33.658702 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ee496-cd63-49d7-822a-5404129b87f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:33.662052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ee496-cd63-49d7-822a-5404129b87f8 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:33.674758 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1285/2615] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:33 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:37.402519 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a0631-d880-46c1-aa39-3f02a10b9cb0 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:37.448421 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:37.479583 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a0631-d880-46c1-aa39-3f02a10b9cb0 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:37.486046 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9284ed-e74b-4d6c-a83c-c50aec9bbdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:37.486960 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9284ed-e74b-4d6c-a83c-c50aec9bbdac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:37.487379 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9284ed-e74b-4d6c-a83c-c50aec9bbdac admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:37.487797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9284ed-e74b-4d6c-a83c-c50aec9bbdac admin admin] PATH_INFO: `/v3/users/605cfc72dd614e7ebe12811a0e48c200` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:37.489453 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9284ed-e74b-4d6c-a83c-c50aec9bbdac admin admin] RBAC: Authorizing `identity:delete_user(user_id=605cfc72dd614e7ebe12811a0e48c200)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:37.492611 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9284ed-e74b-4d6c-a83c-c50aec9bbdac admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:37.546085 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ee9284ed-e74b-4d6c-a83c-c50aec9bbdac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '605cfc72dd614e7ebe12811a0e48c200'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:37.556462 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ee9284ed-e74b-4d6c-a83c-c50aec9bbdac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '605cfc72dd614e7ebe12811a0e48c200'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:37.566464 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ee9284ed-e74b-4d6c-a83c-c50aec9bbdac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '605cfc72dd614e7ebe12811a0e48c200'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:37.567462 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ee9284ed-e74b-4d6c-a83c-c50aec9bbdac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '605cfc72dd614e7ebe12811a0e48c200'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:37.568306 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ee9284ed-e74b-4d6c-a83c-c50aec9bbdac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '605cfc72dd614e7ebe12811a0e48c200'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:37.589231 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1331/2616] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:37 2022] DELETE /identity/v3/users/605cfc72dd614e7ebe12811a0e48c200 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:37.594331 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e90301-90cf-4d5b-8702-cc4bb7b59587 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:37.594789 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e90301-90cf-4d5b-8702-cc4bb7b59587 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:37.595126 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e90301-90cf-4d5b-8702-cc4bb7b59587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:37.615829 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-f9e90301-90cf-4d5b-8702-cc4bb7b59587 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:37.620730 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f9e90301-90cf-4d5b-8702-cc4bb7b59587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:37.696449 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1286/2617] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:39:37.707806 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9284ed-e74b-4d6c-a83c-c50aec9bbdac admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:37.709296 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ee9284ed-e74b-4d6c-a83c-c50aec9bbdac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:37.788646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9284ed-e74b-4d6c-a83c-c50aec9bbdac admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:37.793907 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f543a-77fb-466b-a5e2-0bd74bbd4b3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:37.794588 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f543a-77fb-466b-a5e2-0bd74bbd4b3c service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:37.794923 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f543a-77fb-466b-a5e2-0bd74bbd4b3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:37.795280 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f543a-77fb-466b-a5e2-0bd74bbd4b3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:37.805680 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f543a-77fb-466b-a5e2-0bd74bbd4b3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:37.807280 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f543a-77fb-466b-a5e2-0bd74bbd4b3c service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:37.818276 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1332/2618] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:39:38.137205 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e90301-90cf-4d5b-8702-cc4bb7b59587 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:38.138658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f9e90301-90cf-4d5b-8702-cc4bb7b59587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:38.153430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e90301-90cf-4d5b-8702-cc4bb7b59587 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:38.160534 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e7dd1-6e2c-42df-84c1-92ff914775e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:38.161514 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e7dd1-6e2c-42df-84c1-92ff914775e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:38.162242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e7dd1-6e2c-42df-84c1-92ff914775e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:38.162365 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6e7dd1-6e2c-42df-84c1-92ff914775e0 admin admin] PATH_INFO: `/v3/projects/8fa009d7e07e47aeb5fa9adbbdabda13` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:38.164093 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e7dd1-6e2c-42df-84c1-92ff914775e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fa009d7e07e47aeb5fa9adbbdabda13)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:38.168015 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6e7dd1-6e2c-42df-84c1-92ff914775e0 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:38.233256 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0b6e7dd1-6e2c-42df-84c1-92ff914775e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fa009d7e07e47aeb5fa9adbbdabda13', 'request_id': 'req-0b6e7dd1-6e2c-42df-84c1-92ff914775e0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:38.239538 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0b6e7dd1-6e2c-42df-84c1-92ff914775e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fa009d7e07e47aeb5fa9adbbdabda13', 'request_id': 'req-0b6e7dd1-6e2c-42df-84c1-92ff914775e0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:38.249660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0b6e7dd1-6e2c-42df-84c1-92ff914775e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fa009d7e07e47aeb5fa9adbbdabda13', 'request_id': 'req-0b6e7dd1-6e2c-42df-84c1-92ff914775e0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:38.263626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0b6e7dd1-6e2c-42df-84c1-92ff914775e0 admin admin] The token cache is being invalidate because project 8fa009d7e07e47aeb5fa9adbbdabda13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:39:38.263919 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0b6e7dd1-6e2c-42df-84c1-92ff914775e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:38.264626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-0b6e7dd1-6e2c-42df-84c1-92ff914775e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:38.266269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1287/2619] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:38 2022] DELETE /identity/v3/projects/8fa009d7e07e47aeb5fa9adbbdabda13 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:39:38.270451 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f543a-77fb-466b-a5e2-0bd74bbd4b3c service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:38.281140 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f543a-77fb-466b-a5e2-0bd74bbd4b3c service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:38.342896 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae68b64-9fa1-49d3-904e-aa9b86d0c555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:38.343543 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae68b64-9fa1-49d3-904e-aa9b86d0c555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:38.343877 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae68b64-9fa1-49d3-904e-aa9b86d0c555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:38.344184 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae68b64-9fa1-49d3-904e-aa9b86d0c555 admin admin] PATH_INFO: `/v3/users/13d3ac0a782643a18676807770f562ff` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:38.345310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae68b64-9fa1-49d3-904e-aa9b86d0c555 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13d3ac0a782643a18676807770f562ff)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:38.347238 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae68b64-9fa1-49d3-904e-aa9b86d0c555 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:38.348889 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:38.415563 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7ae68b64-9fa1-49d3-904e-aa9b86d0c555 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13d3ac0a782643a18676807770f562ff'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:38.428200 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7ae68b64-9fa1-49d3-904e-aa9b86d0c555 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13d3ac0a782643a18676807770f562ff'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:38.446838 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7ae68b64-9fa1-49d3-904e-aa9b86d0c555 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13d3ac0a782643a18676807770f562ff'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:38.447494 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7ae68b64-9fa1-49d3-904e-aa9b86d0c555 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13d3ac0a782643a18676807770f562ff'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:38.448144 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7ae68b64-9fa1-49d3-904e-aa9b86d0c555 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13d3ac0a782643a18676807770f562ff'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:38.471464 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1333/2620] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:38 2022] DELETE /identity/v3/users/13d3ac0a782643a18676807770f562ff => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:39:38.767124 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e7dd1-6e2c-42df-84c1-92ff914775e0 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:38.768642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0b6e7dd1-6e2c-42df-84c1-92ff914775e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:38.783181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6e7dd1-6e2c-42df-84c1-92ff914775e0 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:38.866592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb2e48-3ffa-4e49-87c3-1932919eafe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:38.867818 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb2e48-3ffa-4e49-87c3-1932919eafe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:38.868192 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb2e48-3ffa-4e49-87c3-1932919eafe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:38.868539 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb2e48-3ffa-4e49-87c3-1932919eafe7 admin admin] PATH_INFO: `/v3/projects/068c287023984e35aceaf3f3f88acfae` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:38.869749 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb2e48-3ffa-4e49-87c3-1932919eafe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=068c287023984e35aceaf3f3f88acfae)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:38.871725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb2e48-3ffa-4e49-87c3-1932919eafe7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:38.873629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:38.937881 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-6aeb2e48-3ffa-4e49-87c3-1932919eafe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '068c287023984e35aceaf3f3f88acfae', 'request_id': 'req-6aeb2e48-3ffa-4e49-87c3-1932919eafe7'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:38.953910 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-6aeb2e48-3ffa-4e49-87c3-1932919eafe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '068c287023984e35aceaf3f3f88acfae', 'request_id': 'req-6aeb2e48-3ffa-4e49-87c3-1932919eafe7'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:38.956336 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae68b64-9fa1-49d3-904e-aa9b86d0c555 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:38.958587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7ae68b64-9fa1-49d3-904e-aa9b86d0c555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:38.963239 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-6aeb2e48-3ffa-4e49-87c3-1932919eafe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '068c287023984e35aceaf3f3f88acfae', 'request_id': 'req-6aeb2e48-3ffa-4e49-87c3-1932919eafe7'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:38.976507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-6aeb2e48-3ffa-4e49-87c3-1932919eafe7 admin admin] The token cache is being invalidate because project 068c287023984e35aceaf3f3f88acfae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:39:38.977060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-6aeb2e48-3ffa-4e49-87c3-1932919eafe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:38.977846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-6aeb2e48-3ffa-4e49-87c3-1932919eafe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:38.979727 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1288/2621] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:38 2022] DELETE /identity/v3/projects/068c287023984e35aceaf3f3f88acfae => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:38.985106 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b40d8-c9d5-4d18-90ad-be361c1c8d93 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:38.985501 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b40d8-c9d5-4d18-90ad-be361c1c8d93 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:38.985879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b40d8-c9d5-4d18-90ad-be361c1c8d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:39.007650 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-177b40d8-c9d5-4d18-90ad-be361c1c8d93 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:39.008979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-177b40d8-c9d5-4d18-90ad-be361c1c8d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:39.023692 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae68b64-9fa1-49d3-904e-aa9b86d0c555 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:39.080497 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1289/2622] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:39:39.090087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87af73-61e8-45a9-9359-04b4ff9d284e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:39.090586 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87af73-61e8-45a9-9359-04b4ff9d284e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:39.091067 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87af73-61e8-45a9-9359-04b4ff9d284e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:39.101847 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8c626-c167-4714-9711-45abf6a8617c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:39.102833 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8c626-c167-4714-9711-45abf6a8617c service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:39.104128 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8c626-c167-4714-9711-45abf6a8617c service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:39.104502 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8c626-c167-4714-9711-45abf6a8617c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:39.106619 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-51b8c626-c167-4714-9711-45abf6a8617c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:39.116678 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ef87af73-61e8-45a9-9359-04b4ff9d284e None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:39.118557 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ef87af73-61e8-45a9-9359-04b4ff9d284e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:39.125152 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8c626-c167-4714-9711-45abf6a8617c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:39.127725 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b8c626-c167-4714-9711-45abf6a8617c service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:39.130750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1290/2623] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:39:39.136152 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87af73-61e8-45a9-9359-04b4ff9d284e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:39.143895 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1334/2624] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:39:39.149133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87af73-61e8-45a9-9359-04b4ff9d284e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:39.155349 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a22a5-064a-4e8c-8eb0-135427efc581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:39.155982 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a22a5-064a-4e8c-8eb0-135427efc581 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:39.156262 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a22a5-064a-4e8c-8eb0-135427efc581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:39.156526 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a22a5-064a-4e8c-8eb0-135427efc581 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:39.157173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a22a5-064a-4e8c-8eb0-135427efc581 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:39:39.157475 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a22a5-064a-4e8c-8eb0-135427efc581 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:39.159108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a22a5-064a-4e8c-8eb0-135427efc581 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:39.171879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1291/2625] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:39:39.176283 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8c626-c167-4714-9711-45abf6a8617c service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:39.187177 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b8c626-c167-4714-9711-45abf6a8617c service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:39.192020 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c43b9a-df29-4546-ae29-4f768b995c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:39.192640 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c43b9a-df29-4546-ae29-4f768b995c69 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:39.192923 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c43b9a-df29-4546-ae29-4f768b995c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:39.193197 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c43b9a-df29-4546-ae29-4f768b995c69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:39.193845 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c43b9a-df29-4546-ae29-4f768b995c69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:39.195780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c43b9a-df29-4546-ae29-4f768b995c69 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:39.226650 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1335/2626] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:39 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:39.231922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a22a5-064a-4e8c-8eb0-135427efc581 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:39.249668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a22a5-064a-4e8c-8eb0-135427efc581 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:39.328068 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4cb13-7d57-4930-8cb3-fee93fe9fc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:39.329141 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4cb13-7d57-4930-8cb3-fee93fe9fc58 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:39.329543 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4cb13-7d57-4930-8cb3-fee93fe9fc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:39.330004 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4cb13-7d57-4930-8cb3-fee93fe9fc58 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:39.331162 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4cb13-7d57-4930-8cb3-fee93fe9fc58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:39.334730 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4cb13-7d57-4930-8cb3-fee93fe9fc58 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:39.359654 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:39.369405 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1292/2627] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:39 2022] POST /identity/v3/users => generated 295 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:39.372542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c43b9a-df29-4546-ae29-4f768b995c69 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:39.389702 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c43b9a-df29-4546-ae29-4f768b995c69 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:39.397662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003155b-e429-432a-aeda-09c7b0837672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:39.399206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003155b-e429-432a-aeda-09c7b0837672 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:39.399670 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003155b-e429-432a-aeda-09c7b0837672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:39.400165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003155b-e429-432a-aeda-09c7b0837672 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:39.401261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a003155b-e429-432a-aeda-09c7b0837672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:39.404143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a003155b-e429-432a-aeda-09c7b0837672 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:39.423978 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1336/2628] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:39.428584 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4cb13-7d57-4930-8cb3-fee93fe9fc58 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:39.444546 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4cb13-7d57-4930-8cb3-fee93fe9fc58 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:39.452351 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9755abf-8f2e-46ce-afbf-b93df8174dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:39.453301 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9755abf-8f2e-46ce-afbf-b93df8174dcc admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:39.453778 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9755abf-8f2e-46ce-afbf-b93df8174dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:39.454201 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9755abf-8f2e-46ce-afbf-b93df8174dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:39.455266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9755abf-8f2e-46ce-afbf-b93df8174dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:39.458085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9755abf-8f2e-46ce-afbf-b93df8174dcc admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:39.486206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1293/2629] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:39.488491 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003155b-e429-432a-aeda-09c7b0837672 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:39.499601 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:39.508603 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003155b-e429-432a-aeda-09c7b0837672 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:39.522322 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127150f-b38f-4673-aacd-b4f0706828c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:39.524704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127150f-b38f-4673-aacd-b4f0706828c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:39.525599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127150f-b38f-4673-aacd-b4f0706828c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:39.525995 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127150f-b38f-4673-aacd-b4f0706828c3 admin admin] PATH_INFO: `/v3/projects/90a09031c5ba4c2492aeee2592021be7/users/7436369d8f294fbfaf331a2f80d5d602/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:39.555839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127150f-b38f-4673-aacd-b4f0706828c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90a09031c5ba4c2492aeee2592021be7, user_id=7436369d8f294fbfaf331a2f80d5d602, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:39.558940 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127150f-b38f-4673-aacd-b4f0706828c3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:39.578567 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1337/2630] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:39 2022] PUT /identity/v3/projects/90a09031c5ba4c2492aeee2592021be7/users/7436369d8f294fbfaf331a2f80d5d602/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:39.596824 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81e48a-f36a-4263-a9d8-231344fe4407 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:39.597676 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81e48a-f36a-4263-a9d8-231344fe4407 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:39.598085 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81e48a-f36a-4263-a9d8-231344fe4407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:39.630894 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-cf81e48a-f36a-4263-a9d8-231344fe4407 None None] MFA Rules not processed for user `7436369d8f294fbfaf331a2f80d5d602`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:39.632435 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-cf81e48a-f36a-4263-a9d8-231344fe4407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:39.748099 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1294/2631] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:39 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:39.752363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127150f-b38f-4673-aacd-b4f0706828c3 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:39.851328 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127150f-b38f-4673-aacd-b4f0706828c3 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:39.859665 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea2be5-4ead-4208-8a66-e8801536cb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:39.860681 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea2be5-4ead-4208-8a66-e8801536cb76 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:39.861193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea2be5-4ead-4208-8a66-e8801536cb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:39.861587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea2be5-4ead-4208-8a66-e8801536cb76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:39.862641 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea2be5-4ead-4208-8a66-e8801536cb76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:39.866813 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea2be5-4ead-4208-8a66-e8801536cb76 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:39.932310 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1338/2632] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:39 2022] POST /identity/v3/projects => generated 389 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:39.939231 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81e48a-f36a-4263-a9d8-231344fe4407 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:39.968491 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81e48a-f36a-4263-a9d8-231344fe4407 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:40.058119 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a9e93-0684-46bd-9930-9e9237636fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:40.059170 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a9e93-0684-46bd-9930-9e9237636fcb admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:40.059863 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a9e93-0684-46bd-9930-9e9237636fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:40.060430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a9e93-0684-46bd-9930-9e9237636fcb admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:40.061609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a9e93-0684-46bd-9930-9e9237636fcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:40.065356 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a9e93-0684-46bd-9930-9e9237636fcb admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:40.103573 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1295/2633] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:39 2022] POST /identity/v3/users => generated 301 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:40.108350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea2be5-4ead-4208-8a66-e8801536cb76 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:40.125064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea2be5-4ead-4208-8a66-e8801536cb76 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:40.131367 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469a36c-4d15-4ed6-8370-643361bda5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:40.132257 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469a36c-4d15-4ed6-8370-643361bda5df admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:40.132727 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469a36c-4d15-4ed6-8370-643361bda5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:40.133639 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469a36c-4d15-4ed6-8370-643361bda5df admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:40.134202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469a36c-4d15-4ed6-8370-643361bda5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:40.135983 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469a36c-4d15-4ed6-8370-643361bda5df admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:40.153399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a9e93-0684-46bd-9930-9e9237636fcb admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:40.165566 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a9e93-0684-46bd-9930-9e9237636fcb admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:40.166439 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1339/2634] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:40.170972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4cc77-bc92-40f3-ae40-51f98bee93b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:40.171646 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4cc77-bc92-40f3-ae40-51f98bee93b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:40.171923 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4cc77-bc92-40f3-ae40-51f98bee93b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:40.172232 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4cc77-bc92-40f3-ae40-51f98bee93b2 admin admin] PATH_INFO: `/v3/projects/d4bdc06f5eab4fd1993651924f65d9bb/users/9077ee2cb3eb449aa3e443f971c47639/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:40.188207 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4cc77-bc92-40f3-ae40-51f98bee93b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4bdc06f5eab4fd1993651924f65d9bb, user_id=9077ee2cb3eb449aa3e443f971c47639, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:40.190867 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4cc77-bc92-40f3-ae40-51f98bee93b2 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:40.205578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1296/2635] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:40 2022] PUT /identity/v3/projects/d4bdc06f5eab4fd1993651924f65d9bb/users/9077ee2cb3eb449aa3e443f971c47639/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:40.209439 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469a36c-4d15-4ed6-8370-643361bda5df admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:40.221272 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469a36c-4d15-4ed6-8370-643361bda5df admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:40.284345 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a919c0-fd29-4773-bf7d-321b5496091b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:40.292164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a919c0-fd29-4773-bf7d-321b5496091b admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:40.292164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a919c0-fd29-4773-bf7d-321b5496091b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:40.292164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a919c0-fd29-4773-bf7d-321b5496091b admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:40.292164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a919c0-fd29-4773-bf7d-321b5496091b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:40.292164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a919c0-fd29-4773-bf7d-321b5496091b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:40.299475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1340/2636] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:40.306670 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4cc77-bc92-40f3-ae40-51f98bee93b2 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:40.325283 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4cc77-bc92-40f3-ae40-51f98bee93b2 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:40.333158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95e279-77cb-49b6-a784-897a2c554593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:40.334216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95e279-77cb-49b6-a784-897a2c554593 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:40.334682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95e279-77cb-49b6-a784-897a2c554593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:40.335150 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95e279-77cb-49b6-a784-897a2c554593 admin admin] PATH_INFO: `/v3/projects/d4bdc06f5eab4fd1993651924f65d9bb/users/9077ee2cb3eb449aa3e443f971c47639/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:40.338361 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95e279-77cb-49b6-a784-897a2c554593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4bdc06f5eab4fd1993651924f65d9bb, user_id=9077ee2cb3eb449aa3e443f971c47639, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:40.343274 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95e279-77cb-49b6-a784-897a2c554593 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:40.360145 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1297/2637] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:40 2022] PUT /identity/v3/projects/d4bdc06f5eab4fd1993651924f65d9bb/users/9077ee2cb3eb449aa3e443f971c47639/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:40.430501 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee99ca-ce2e-4311-b996-eb3cb727f106 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:40.431085 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee99ca-ce2e-4311-b996-eb3cb727f106 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:40.431203 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee99ca-ce2e-4311-b996-eb3cb727f106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:40.481102 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-72ee99ca-ce2e-4311-b996-eb3cb727f106 None None] MFA Rules not processed for user `9077ee2cb3eb449aa3e443f971c47639`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:40.483175 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-72ee99ca-ce2e-4311-b996-eb3cb727f106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:40.533184 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:40.582634 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1341/2638] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:40 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:39:40.592233 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95e279-77cb-49b6-a784-897a2c554593 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:40.594279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8b95e279-77cb-49b6-a784-897a2c554593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:40.639407 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:40.673189 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95e279-77cb-49b6-a784-897a2c554593 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:40.678436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c165d-034f-45f9-b501-4f707d150374 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:40.679095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c165d-034f-45f9-b501-4f707d150374 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:40.679434 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c165d-034f-45f9-b501-4f707d150374 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:40.679630 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c165d-034f-45f9-b501-4f707d150374 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:40.691319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c165d-034f-45f9-b501-4f707d150374 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:40.692906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c165d-034f-45f9-b501-4f707d150374 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:40.703975 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1298/2639] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:40 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:39:40.858247 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee99ca-ce2e-4311-b996-eb3cb727f106 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:40.931356 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee99ca-ce2e-4311-b996-eb3cb727f106 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:40.936470 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb46d2-ccff-472e-970c-1dc7f1689e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:40.937196 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb46d2-ccff-472e-970c-1dc7f1689e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:40.937485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb46d2-ccff-472e-970c-1dc7f1689e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:40.938021 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb46d2-ccff-472e-970c-1dc7f1689e88 admin admin] PATH_INFO: `/v3/users/7436369d8f294fbfaf331a2f80d5d602` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:40.940347 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb46d2-ccff-472e-970c-1dc7f1689e88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7436369d8f294fbfaf331a2f80d5d602)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:40.942278 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb46d2-ccff-472e-970c-1dc7f1689e88 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:41.002647 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9dcb46d2-ccff-472e-970c-1dc7f1689e88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7436369d8f294fbfaf331a2f80d5d602'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:41.014688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9dcb46d2-ccff-472e-970c-1dc7f1689e88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7436369d8f294fbfaf331a2f80d5d602'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:41.026575 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9dcb46d2-ccff-472e-970c-1dc7f1689e88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7436369d8f294fbfaf331a2f80d5d602'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:41.027704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9dcb46d2-ccff-472e-970c-1dc7f1689e88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7436369d8f294fbfaf331a2f80d5d602'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:41.028734 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9dcb46d2-ccff-472e-970c-1dc7f1689e88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7436369d8f294fbfaf331a2f80d5d602'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:41.060724 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1342/2640] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:40 2022] DELETE /identity/v3/users/7436369d8f294fbfaf331a2f80d5d602 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:39:41.068722 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6558a71-21a8-400f-9ea9-51d1fe5fad29 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:41.069274 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6558a71-21a8-400f-9ea9-51d1fe5fad29 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:41.070049 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6558a71-21a8-400f-9ea9-51d1fe5fad29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:41.102455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b6558a71-21a8-400f-9ea9-51d1fe5fad29 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:41.103754 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b6558a71-21a8-400f-9ea9-51d1fe5fad29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:41.183970 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1299/2641] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:41.194937 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb46d2-ccff-472e-970c-1dc7f1689e88 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:41.198713 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9dcb46d2-ccff-472e-970c-1dc7f1689e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:41.295451 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb46d2-ccff-472e-970c-1dc7f1689e88 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:41.309958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d4b41-7f88-42ad-9474-6280ca595838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:41.310613 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d4b41-7f88-42ad-9474-6280ca595838 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:41.311014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d4b41-7f88-42ad-9474-6280ca595838 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:41.311297 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d4b41-7f88-42ad-9474-6280ca595838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:41.322142 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d4b41-7f88-42ad-9474-6280ca595838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:41.323638 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d4b41-7f88-42ad-9474-6280ca595838 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:41.328587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:41.344647 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1343/2642] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:41.746423 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6558a71-21a8-400f-9ea9-51d1fe5fad29 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:41.748869 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b6558a71-21a8-400f-9ea9-51d1fe5fad29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:41.780209 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6558a71-21a8-400f-9ea9-51d1fe5fad29 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:41.793285 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce838024-9556-40db-8dcd-c18e501aeac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:41.794558 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce838024-9556-40db-8dcd-c18e501aeac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:41.795306 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce838024-9556-40db-8dcd-c18e501aeac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:41.795560 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce838024-9556-40db-8dcd-c18e501aeac8 admin admin] PATH_INFO: `/v3/projects/90a09031c5ba4c2492aeee2592021be7` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:41.798025 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce838024-9556-40db-8dcd-c18e501aeac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90a09031c5ba4c2492aeee2592021be7)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:41.802924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce838024-9556-40db-8dcd-c18e501aeac8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:41.910670 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ce838024-9556-40db-8dcd-c18e501aeac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90a09031c5ba4c2492aeee2592021be7', 'request_id': 'req-ce838024-9556-40db-8dcd-c18e501aeac8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:41.920619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ce838024-9556-40db-8dcd-c18e501aeac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90a09031c5ba4c2492aeee2592021be7', 'request_id': 'req-ce838024-9556-40db-8dcd-c18e501aeac8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:41.935276 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ce838024-9556-40db-8dcd-c18e501aeac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90a09031c5ba4c2492aeee2592021be7', 'request_id': 'req-ce838024-9556-40db-8dcd-c18e501aeac8'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:41.971542 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ce838024-9556-40db-8dcd-c18e501aeac8 admin admin] The token cache is being invalidate because project 90a09031c5ba4c2492aeee2592021be7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:39:41.972162 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ce838024-9556-40db-8dcd-c18e501aeac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:41.973258 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ce838024-9556-40db-8dcd-c18e501aeac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:41.979741 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1300/2643] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:41 2022] DELETE /identity/v3/projects/90a09031c5ba4c2492aeee2592021be7 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:39:41.985118 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d4b41-7f88-42ad-9474-6280ca595838 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:42.014030 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d4b41-7f88-42ad-9474-6280ca595838 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:42.184327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129b0b7-abc7-44dc-a510-4a43d3a9accd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:42.186958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129b0b7-abc7-44dc-a510-4a43d3a9accd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:42.187708 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129b0b7-abc7-44dc-a510-4a43d3a9accd admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:42.188389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129b0b7-abc7-44dc-a510-4a43d3a9accd admin admin] PATH_INFO: `/v3/users/9077ee2cb3eb449aa3e443f971c47639` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:42.190805 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129b0b7-abc7-44dc-a510-4a43d3a9accd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9077ee2cb3eb449aa3e443f971c47639)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:42.195222 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129b0b7-abc7-44dc-a510-4a43d3a9accd admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:42.295797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d129b0b7-abc7-44dc-a510-4a43d3a9accd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9077ee2cb3eb449aa3e443f971c47639'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:42.308683 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d129b0b7-abc7-44dc-a510-4a43d3a9accd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9077ee2cb3eb449aa3e443f971c47639'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:42.321026 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d129b0b7-abc7-44dc-a510-4a43d3a9accd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9077ee2cb3eb449aa3e443f971c47639'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:42.321662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d129b0b7-abc7-44dc-a510-4a43d3a9accd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9077ee2cb3eb449aa3e443f971c47639'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:42.322272 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-d129b0b7-abc7-44dc-a510-4a43d3a9accd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9077ee2cb3eb449aa3e443f971c47639'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:42.349790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1344/2644] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:41 2022] DELETE /identity/v3/users/9077ee2cb3eb449aa3e443f971c47639 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:39:42.922312 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce838024-9556-40db-8dcd-c18e501aeac8 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:42.924430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ce838024-9556-40db-8dcd-c18e501aeac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:42.944821 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce838024-9556-40db-8dcd-c18e501aeac8 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:43.031552 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:43.044174 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d4777-250c-4a7e-b861-bcceee16914c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:43.046127 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d4777-250c-4a7e-b861-bcceee16914c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:43.046708 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d4777-250c-4a7e-b861-bcceee16914c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:43.047436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d4777-250c-4a7e-b861-bcceee16914c admin admin] PATH_INFO: `/v3/projects/d4bdc06f5eab4fd1993651924f65d9bb` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:43.064951 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d4777-250c-4a7e-b861-bcceee16914c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4bdc06f5eab4fd1993651924f65d9bb)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:43.068979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d4777-250c-4a7e-b861-bcceee16914c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:43.204357 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d15d4777-250c-4a7e-b861-bcceee16914c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4bdc06f5eab4fd1993651924f65d9bb', 'request_id': 'req-d15d4777-250c-4a7e-b861-bcceee16914c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:43.222105 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d15d4777-250c-4a7e-b861-bcceee16914c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4bdc06f5eab4fd1993651924f65d9bb', 'request_id': 'req-d15d4777-250c-4a7e-b861-bcceee16914c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:43.239730 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d15d4777-250c-4a7e-b861-bcceee16914c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4bdc06f5eab4fd1993651924f65d9bb', 'request_id': 'req-d15d4777-250c-4a7e-b861-bcceee16914c'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:43.263491 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d15d4777-250c-4a7e-b861-bcceee16914c admin admin] The token cache is being invalidate because project d4bdc06f5eab4fd1993651924f65d9bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:39:43.263787 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d15d4777-250c-4a7e-b861-bcceee16914c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:43.264439 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d15d4777-250c-4a7e-b861-bcceee16914c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:43.266854 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1301/2645] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:42 2022] DELETE /identity/v3/projects/d4bdc06f5eab4fd1993651924f65d9bb => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:43.273307 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53290f-f7df-4fae-83a2-af953dac8e5e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:43.275706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53290f-f7df-4fae-83a2-af953dac8e5e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:43.276244 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53290f-f7df-4fae-83a2-af953dac8e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:43.328735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-cc53290f-f7df-4fae-83a2-af953dac8e5e None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:43.330301 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-cc53290f-f7df-4fae-83a2-af953dac8e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:43.467796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1345/2646] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:39:43.477276 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94e0c3-8f26-4877-834e-05341a8201f3 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:43.477991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94e0c3-8f26-4877-834e-05341a8201f3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:43.478660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee94e0c3-8f26-4877-834e-05341a8201f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:43.482933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53290f-f7df-4fae-83a2-af953dac8e5e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:43.487350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-cc53290f-f7df-4fae-83a2-af953dac8e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:43.530454 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:43.542221 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ee94e0c3-8f26-4877-834e-05341a8201f3 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:43.545344 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ee94e0c3-8f26-4877-834e-05341a8201f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:43.589252 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1302/2647] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:43.593199 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94e0c3-8f26-4877-834e-05341a8201f3 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:43.606485 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee94e0c3-8f26-4877-834e-05341a8201f3 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:43.612774 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53290f-f7df-4fae-83a2-af953dac8e5e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:43.618155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a409f-da13-4be0-bca3-e407b890d7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:43.618834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939d38f-c328-41fe-9a09-2f6dc952285e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0e659eee5e4a9da8d3d7337cb73798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:43.619407 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a409f-da13-4be0-bca3-e407b890d7ef admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:43.620153 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a409f-da13-4be0-bca3-e407b890d7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:43.620341 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939d38f-c328-41fe-9a09-2f6dc952285e service placement] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:43.620807 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939d38f-c328-41fe-9a09-2f6dc952285e service placement] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:43.621275 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939d38f-c328-41fe-9a09-2f6dc952285e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:43.621725 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a409f-da13-4be0-bca3-e407b890d7ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:43.623639 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3939d38f-c328-41fe-9a09-2f6dc952285e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:43.623883 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a409f-da13-4be0-bca3-e407b890d7ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:39:43.624528 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a409f-da13-4be0-bca3-e407b890d7ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:43.628024 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a409f-da13-4be0-bca3-e407b890d7ef admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:43.641380 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939d38f-c328-41fe-9a09-2f6dc952285e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:43.643423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939d38f-c328-41fe-9a09-2f6dc952285e service placement] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:43.657483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1346/2648] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:43.683843 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1303/2649] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:39:43.684293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939d38f-c328-41fe-9a09-2f6dc952285e service placement] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:43.684293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3939d38f-c328-41fe-9a09-2f6dc952285e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:43.688733 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a409f-da13-4be0-bca3-e407b890d7ef admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:43.703144 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939d38f-c328-41fe-9a09-2f6dc952285e service placement] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:43.712424 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676a06f-d684-4d2b-bb2d-7a9e8b4fdf5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:43.713108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a409f-da13-4be0-bca3-e407b890d7ef admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:43.713507 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676a06f-d684-4d2b-bb2d-7a9e8b4fdf5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:43.714015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676a06f-d684-4d2b-bb2d-7a9e8b4fdf5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:43.714531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676a06f-d684-4d2b-bb2d-7a9e8b4fdf5a admin admin] PATH_INFO: `/v3/users/adc5854ed52042369a9b153af7da4cfb` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:43.716761 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6676a06f-d684-4d2b-bb2d-7a9e8b4fdf5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=adc5854ed52042369a9b153af7da4cfb)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:43.720581 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6676a06f-d684-4d2b-bb2d-7a9e8b4fdf5a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:43.726471 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19bb1c-2051-46cf-99df-6d3a83d12d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:43.728051 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19bb1c-2051-46cf-99df-6d3a83d12d73 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:43.728636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19bb1c-2051-46cf-99df-6d3a83d12d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:43.731488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19bb1c-2051-46cf-99df-6d3a83d12d73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:43.733084 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19bb1c-2051-46cf-99df-6d3a83d12d73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:43.736946 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19bb1c-2051-46cf-99df-6d3a83d12d73 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:43.795259 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1304/2650] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:43 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:43.799109 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6676a06f-d684-4d2b-bb2d-7a9e8b4fdf5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adc5854ed52042369a9b153af7da4cfb'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:43.800774 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19bb1c-2051-46cf-99df-6d3a83d12d73 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:43.812186 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6676a06f-d684-4d2b-bb2d-7a9e8b4fdf5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adc5854ed52042369a9b153af7da4cfb'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:43.826189 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6676a06f-d684-4d2b-bb2d-7a9e8b4fdf5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adc5854ed52042369a9b153af7da4cfb'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:43.827417 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6676a06f-d684-4d2b-bb2d-7a9e8b4fdf5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adc5854ed52042369a9b153af7da4cfb'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:43.828476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-6676a06f-d684-4d2b-bb2d-7a9e8b4fdf5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adc5854ed52042369a9b153af7da4cfb'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:43.881136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1347/2651] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:43 2022] DELETE /identity/v3/users/adc5854ed52042369a9b153af7da4cfb => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:43.921885 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19bb1c-2051-46cf-99df-6d3a83d12d73 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:44.024475 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:44.035376 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65c6b5-a865-4b2c-98f0-cb3f34263a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:44.036290 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65c6b5-a865-4b2c-98f0-cb3f34263a70 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:44.036750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65c6b5-a865-4b2c-98f0-cb3f34263a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:44.037282 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65c6b5-a865-4b2c-98f0-cb3f34263a70 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:44.038433 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65c6b5-a865-4b2c-98f0-cb3f34263a70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:44.040987 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65c6b5-a865-4b2c-98f0-cb3f34263a70 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:44.082627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1305/2652] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:43 2022] POST /identity/v3/users => generated 293 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:39:44.083924 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676a06f-d684-4d2b-bb2d-7a9e8b4fdf5a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:44.085430 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6676a06f-d684-4d2b-bb2d-7a9e8b4fdf5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:44.099595 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676a06f-d684-4d2b-bb2d-7a9e8b4fdf5a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:44.106158 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a8140-3707-4a09-bf8d-e6d910fc83bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:44.106784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a8140-3707-4a09-bf8d-e6d910fc83bd admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:44.107070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a8140-3707-4a09-bf8d-e6d910fc83bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:44.107354 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a8140-3707-4a09-bf8d-e6d910fc83bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:44.108124 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a8140-3707-4a09-bf8d-e6d910fc83bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:44.110054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a8140-3707-4a09-bf8d-e6d910fc83bd admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:44.121956 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1348/2653] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:44.127956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65c6b5-a865-4b2c-98f0-cb3f34263a70 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:44.135397 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a8140-3707-4a09-bf8d-e6d910fc83bd admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:44.138554 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-956a8140-3707-4a09-bf8d-e6d910fc83bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:44.145113 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65c6b5-a865-4b2c-98f0-cb3f34263a70 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:44.154296 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ea667-8102-4cdc-9f22-10dd890ce99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:44.155464 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ea667-8102-4cdc-9f22-10dd890ce99a admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:44.155899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ea667-8102-4cdc-9f22-10dd890ce99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:44.156347 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ea667-8102-4cdc-9f22-10dd890ce99a admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:44.157358 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ea667-8102-4cdc-9f22-10dd890ce99a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:44.158505 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a8140-3707-4a09-bf8d-e6d910fc83bd admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:44.160931 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ea667-8102-4cdc-9f22-10dd890ce99a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:44.164456 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d502b6e-7fa6-4790-b284-ff415d15d0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:44.165570 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d502b6e-7fa6-4790-b284-ff415d15d0c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:44.165865 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d502b6e-7fa6-4790-b284-ff415d15d0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:44.166147 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d502b6e-7fa6-4790-b284-ff415d15d0c2 admin admin] PATH_INFO: `/v3/projects/544c78c5c2c44219a7f236b24a107da4` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:44.167525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d502b6e-7fa6-4790-b284-ff415d15d0c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=544c78c5c2c44219a7f236b24a107da4)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:44.169592 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d502b6e-7fa6-4790-b284-ff415d15d0c2 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:44.185243 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1306/2654] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:44.186946 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ea667-8102-4cdc-9f22-10dd890ce99a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:44.204901 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ea667-8102-4cdc-9f22-10dd890ce99a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:44.212809 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd607824-f85d-485d-be37-ed6bd6bd16bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:44.213788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd607824-f85d-485d-be37-ed6bd6bd16bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:44.214211 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd607824-f85d-485d-be37-ed6bd6bd16bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:44.214650 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd607824-f85d-485d-be37-ed6bd6bd16bc admin admin] PATH_INFO: `/v3/projects/44354b71c294460a9eac3b94241ffb55/users/1c8ccb1d8a104314a3bdf30106c20ef7/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:44.226998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9d502b6e-7fa6-4790-b284-ff415d15d0c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '544c78c5c2c44219a7f236b24a107da4', 'request_id': 'req-9d502b6e-7fa6-4790-b284-ff415d15d0c2'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:44.235488 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9d502b6e-7fa6-4790-b284-ff415d15d0c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '544c78c5c2c44219a7f236b24a107da4', 'request_id': 'req-9d502b6e-7fa6-4790-b284-ff415d15d0c2'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:44.238869 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd607824-f85d-485d-be37-ed6bd6bd16bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=44354b71c294460a9eac3b94241ffb55, user_id=1c8ccb1d8a104314a3bdf30106c20ef7, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:44.240643 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:44.242966 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9d502b6e-7fa6-4790-b284-ff415d15d0c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '544c78c5c2c44219a7f236b24a107da4', 'request_id': 'req-9d502b6e-7fa6-4790-b284-ff415d15d0c2'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:44.244736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd607824-f85d-485d-be37-ed6bd6bd16bc admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:44.261486 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9d502b6e-7fa6-4790-b284-ff415d15d0c2 admin admin] The token cache is being invalidate because project 544c78c5c2c44219a7f236b24a107da4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:39:44.262111 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9d502b6e-7fa6-4790-b284-ff415d15d0c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:44.263109 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-9d502b6e-7fa6-4790-b284-ff415d15d0c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:44.264986 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1349/2655] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:44 2022] DELETE /identity/v3/projects/544c78c5c2c44219a7f236b24a107da4 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:44.266898 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1307/2656] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:44 2022] PUT /identity/v3/projects/44354b71c294460a9eac3b94241ffb55/users/1c8ccb1d8a104314a3bdf30106c20ef7/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:44.271224 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00d00d-9663-46f7-a985-04ef7df23f83 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:44.271517 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00d00d-9663-46f7-a985-04ef7df23f83 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:44.272067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00d00d-9663-46f7-a985-04ef7df23f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:44.279330 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8a8be-8314-45ee-aeb3-019d779d6d6e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:44.279699 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8a8be-8314-45ee-aeb3-019d779d6d6e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:44.280123 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8a8be-8314-45ee-aeb3-019d779d6d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:44.295651 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4d00d00d-9663-46f7-a985-04ef7df23f83 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:44.297218 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4d00d00d-9663-46f7-a985-04ef7df23f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:44.320193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-66d8a8be-8314-45ee-aeb3-019d779d6d6e None None] MFA Rules not processed for user `1c8ccb1d8a104314a3bdf30106c20ef7`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:44.321432 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-66d8a8be-8314-45ee-aeb3-019d779d6d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:44.368116 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1350/2657] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:39:44.380981 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a34d4-68c8-4040-b2e6-d117d579844b None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:44.381313 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a34d4-68c8-4040-b2e6-d117d579844b None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:44.381555 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a34d4-68c8-4040-b2e6-d117d579844b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:44.411523 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1308/2658] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:39:44.416508 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-933a34d4-68c8-4040-b2e6-d117d579844b None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:44.417269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8a8be-8314-45ee-aeb3-019d779d6d6e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:44.418252 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-933a34d4-68c8-4040-b2e6-d117d579844b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:44.418414 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-66d8a8be-8314-45ee-aeb3-019d779d6d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:44.430612 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1351/2659] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:39:44.436551 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8a8be-8314-45ee-aeb3-019d779d6d6e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:44.437801 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a34d4-68c8-4040-b2e6-d117d579844b None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:44.443132 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8bd7a-1fca-48a1-bf44-1690827feb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:44.444026 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8bd7a-1fca-48a1-bf44-1690827feb21 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:44.444304 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8bd7a-1fca-48a1-bf44-1690827feb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:44.444571 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8bd7a-1fca-48a1-bf44-1690827feb21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:44.445506 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd8bd7a-1fca-48a1-bf44-1690827feb21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:44.449196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd8bd7a-1fca-48a1-bf44-1690827feb21 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:44.457518 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a34d4-68c8-4040-b2e6-d117d579844b None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:44.465129 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adf8ad-6834-41d3-ad72-6bf94a11338e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:44.466048 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adf8ad-6834-41d3-ad72-6bf94a11338e admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:44.466656 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adf8ad-6834-41d3-ad72-6bf94a11338e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:44.467071 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adf8ad-6834-41d3-ad72-6bf94a11338e admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:44.469591 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adf8ad-6834-41d3-ad72-6bf94a11338e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:39:44.470073 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adf8ad-6834-41d3-ad72-6bf94a11338e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:44.472516 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adf8ad-6834-41d3-ad72-6bf94a11338e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:44.490269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1352/2660] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:39:44.498144 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1309/2661] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:44 2022] POST /identity/v3/projects => generated 385 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:39:44.498432 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adf8ad-6834-41d3-ad72-6bf94a11338e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:44.499407 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8bd7a-1fca-48a1-bf44-1690827feb21 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:44.511112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adf8ad-6834-41d3-ad72-6bf94a11338e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:44.589819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8bd7a-1fca-48a1-bf44-1690827feb21 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:44.595310 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7479eb-b420-4a93-b3ab-7031c7ca7a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:44.595729 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc049a-bc83-45f9-85d1-68f1f921b3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:44.595926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7479eb-b420-4a93-b3ab-7031c7ca7a10 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:44.596236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7479eb-b420-4a93-b3ab-7031c7ca7a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:44.596377 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc049a-bc83-45f9-85d1-68f1f921b3c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:44.596512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7479eb-b420-4a93-b3ab-7031c7ca7a10 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:44.596798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc049a-bc83-45f9-85d1-68f1f921b3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:44.597074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc049a-bc83-45f9-85d1-68f1f921b3c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:44.597220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7479eb-b420-4a93-b3ab-7031c7ca7a10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:44.597705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc049a-bc83-45f9-85d1-68f1f921b3c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:44.599113 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7479eb-b420-4a93-b3ab-7031c7ca7a10 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:44.599590 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc049a-bc83-45f9-85d1-68f1f921b3c5 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:44.628955 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1310/2662] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:44 2022] POST /identity/v3/users => generated 299 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:39:44.630080 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7479eb-b420-4a93-b3ab-7031c7ca7a10 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:44.636766 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:44.638696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1353/2663] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:44 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:44.643566 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc049a-bc83-45f9-85d1-68f1f921b3c5 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:44.645236 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7479eb-b420-4a93-b3ab-7031c7ca7a10 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:44.712818 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb5a05-b6fc-4837-a1bb-248b460bec1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:44.713320 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb5a05-b6fc-4837-a1bb-248b460bec1f admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:44.713667 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb5a05-b6fc-4837-a1bb-248b460bec1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:44.714130 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb5a05-b6fc-4837-a1bb-248b460bec1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:44.714693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb5a05-b6fc-4837-a1bb-248b460bec1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:44.716434 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb5a05-b6fc-4837-a1bb-248b460bec1f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:44.727878 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1311/2664] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:44.732132 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb5a05-b6fc-4837-a1bb-248b460bec1f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:44.741930 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb5a05-b6fc-4837-a1bb-248b460bec1f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:44.747130 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ab433-1785-4f0a-bfd7-160bf74bc653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:44.747785 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ab433-1785-4f0a-bfd7-160bf74bc653 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:44.748102 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ab433-1785-4f0a-bfd7-160bf74bc653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:44.748380 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ab433-1785-4f0a-bfd7-160bf74bc653 admin admin] PATH_INFO: `/v3/projects/05e8fa04025d4558bacc99faf06f92dc/users/72bdf90ee068430dac22057c6f767fd3/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:44.748963 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:44.757662 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc049a-bc83-45f9-85d1-68f1f921b3c5 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:44.766451 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11779057-7ab2-4eca-9789-44221cabbaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:44.767450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11779057-7ab2-4eca-9789-44221cabbaad admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:44.767908 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11779057-7ab2-4eca-9789-44221cabbaad admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:44.768363 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ab433-1785-4f0a-bfd7-160bf74bc653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05e8fa04025d4558bacc99faf06f92dc, user_id=72bdf90ee068430dac22057c6f767fd3, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:44.768552 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11779057-7ab2-4eca-9789-44221cabbaad admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:44.769440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11779057-7ab2-4eca-9789-44221cabbaad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:44.772901 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11779057-7ab2-4eca-9789-44221cabbaad admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:44.774278 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ab433-1785-4f0a-bfd7-160bf74bc653 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:44.798742 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1312/2665] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:44 2022] PUT /identity/v3/projects/05e8fa04025d4558bacc99faf06f92dc/users/72bdf90ee068430dac22057c6f767fd3/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:39:44.803378 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ab433-1785-4f0a-bfd7-160bf74bc653 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:44.807223 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1354/2666] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:44 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:39:44.808390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11779057-7ab2-4eca-9789-44221cabbaad admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:44.820436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11779057-7ab2-4eca-9789-44221cabbaad admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:44.892050 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733de79-0380-47a3-8642-e23bb5d6fc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:44.892836 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733de79-0380-47a3-8642-e23bb5d6fc54 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:44.893025 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ab433-1785-4f0a-bfd7-160bf74bc653 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:44.893362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733de79-0380-47a3-8642-e23bb5d6fc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:44.893758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733de79-0380-47a3-8642-e23bb5d6fc54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:44.894686 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733de79-0380-47a3-8642-e23bb5d6fc54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:44.896623 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733de79-0380-47a3-8642-e23bb5d6fc54 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:44.901177 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b877585-5f7d-4ffd-92d2-38254447c755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:44.902063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b877585-5f7d-4ffd-92d2-38254447c755 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:44.902340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b877585-5f7d-4ffd-92d2-38254447c755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:44.902613 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b877585-5f7d-4ffd-92d2-38254447c755 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:44.903248 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b877585-5f7d-4ffd-92d2-38254447c755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:44.905012 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b877585-5f7d-4ffd-92d2-38254447c755 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:44.909730 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1355/2667] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:39:44.915120 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733de79-0380-47a3-8642-e23bb5d6fc54 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:44.916471 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1313/2668] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:39:44.920586 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b877585-5f7d-4ffd-92d2-38254447c755 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:44.926984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733de79-0380-47a3-8642-e23bb5d6fc54 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:44.931333 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b877585-5f7d-4ffd-92d2-38254447c755 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:44.933532 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9c3ee-e832-4849-a76a-e944bdcb668e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:44.934311 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9c3ee-e832-4849-a76a-e944bdcb668e admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:44.934981 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9c3ee-e832-4849-a76a-e944bdcb668e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:44.935347 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9c3ee-e832-4849-a76a-e944bdcb668e admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:44.936142 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9c3ee-e832-4849-a76a-e944bdcb668e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:44.936509 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69668a-6af7-42e6-ac02-a66cdfa73bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:44.937269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69668a-6af7-42e6-ac02-a66cdfa73bfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:44.937546 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69668a-6af7-42e6-ac02-a66cdfa73bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:44.938165 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69668a-6af7-42e6-ac02-a66cdfa73bfb admin admin] PATH_INFO: `/v3/projects/05e8fa04025d4558bacc99faf06f92dc/users/72bdf90ee068430dac22057c6f767fd3/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:44.939570 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9c3ee-e832-4849-a76a-e944bdcb668e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:44.941389 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69668a-6af7-42e6-ac02-a66cdfa73bfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=05e8fa04025d4558bacc99faf06f92dc, user_id=72bdf90ee068430dac22057c6f767fd3, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:44.947336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69668a-6af7-42e6-ac02-a66cdfa73bfb admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:44.954168 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1356/2669] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:39:44.959249 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9c3ee-e832-4849-a76a-e944bdcb668e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:44.965824 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1314/2670] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:44 2022] PUT /identity/v3/projects/05e8fa04025d4558bacc99faf06f92dc/users/72bdf90ee068430dac22057c6f767fd3/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:44.973274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9c3ee-e832-4849-a76a-e944bdcb668e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:44.979047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca1b10-286c-4158-b411-29d481605c2d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:44.979340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca1b10-286c-4158-b411-29d481605c2d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:44.979612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca1b10-286c-4158-b411-29d481605c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:45.017382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-2fca1b10-286c-4158-b411-29d481605c2d None None] MFA Rules not processed for user `72bdf90ee068430dac22057c6f767fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:45.019129 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2fca1b10-286c-4158-b411-29d481605c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:45.076319 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7660501-9469-4d6f-ac32-102494399b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:45.077115 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7660501-9469-4d6f-ac32-102494399b25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:45.077489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7660501-9469-4d6f-ac32-102494399b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:45.077851 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7660501-9469-4d6f-ac32-102494399b25 admin admin] PATH_INFO: `/v3/projects/cabee0f9db4b41abba9fced3cf293eb9/users/a004d6c41d1140819d1470f7035f9317/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:45.098814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:45.107082 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7660501-9469-4d6f-ac32-102494399b25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cabee0f9db4b41abba9fced3cf293eb9, user_id=a004d6c41d1140819d1470f7035f9317, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:45.111854 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7660501-9469-4d6f-ac32-102494399b25 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:45.133477 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1357/2671] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:44 2022] PUT /identity/v3/projects/cabee0f9db4b41abba9fced3cf293eb9/users/a004d6c41d1140819d1470f7035f9317/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:39:45.139021 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1315/2672] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:45.149465 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7660501-9469-4d6f-ac32-102494399b25 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:45.151504 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4497d-ad4c-4e32-b853-d9719f1ed588 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:45.151847 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4497d-ad4c-4e32-b853-d9719f1ed588 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:45.152848 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4497d-ad4c-4e32-b853-d9719f1ed588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:45.158669 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e7660501-9469-4d6f-ac32-102494399b25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:45.200519 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-94d4497d-ad4c-4e32-b853-d9719f1ed588 None None] MFA Rules not processed for user `a004d6c41d1140819d1470f7035f9317`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:45.202629 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-94d4497d-ad4c-4e32-b853-d9719f1ed588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:45.255404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7660501-9469-4d6f-ac32-102494399b25 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:45.260640 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277af0f-b7ef-49cd-be7c-1597f706d79a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:45.261589 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277af0f-b7ef-49cd-be7c-1597f706d79a service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:45.262014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277af0f-b7ef-49cd-be7c-1597f706d79a service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:45.262422 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d277af0f-b7ef-49cd-be7c-1597f706d79a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:45.274042 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:45.296432 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1316/2673] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:45.312890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4497d-ad4c-4e32-b853-d9719f1ed588 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:45.314329 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-94d4497d-ad4c-4e32-b853-d9719f1ed588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:45.346541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d277af0f-b7ef-49cd-be7c-1597f706d79a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:45.348924 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d277af0f-b7ef-49cd-be7c-1597f706d79a service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:45.364598 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1358/2674] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:45.392359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4497d-ad4c-4e32-b853-d9719f1ed588 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:45.398998 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5196d9-895c-4d6e-bb09-41af52da5de9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:45.399695 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5196d9-895c-4d6e-bb09-41af52da5de9 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:45.400010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5196d9-895c-4d6e-bb09-41af52da5de9 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:45.400292 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5196d9-895c-4d6e-bb09-41af52da5de9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:45.409285 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5196d9-895c-4d6e-bb09-41af52da5de9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:45.410890 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5196d9-895c-4d6e-bb09-41af52da5de9 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:45.420867 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1317/2675] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:39:46.016305 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277af0f-b7ef-49cd-be7c-1597f706d79a service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:46.098713 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d277af0f-b7ef-49cd-be7c-1597f706d79a service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:46.104437 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54d19f-d0fd-415e-b074-c2bc63bbdaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:46.105225 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54d19f-d0fd-415e-b074-c2bc63bbdaa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:46.105574 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54d19f-d0fd-415e-b074-c2bc63bbdaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:46.105875 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54d19f-d0fd-415e-b074-c2bc63bbdaa4 admin admin] PATH_INFO: `/v3/users/1c8ccb1d8a104314a3bdf30106c20ef7` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:46.107020 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb54d19f-d0fd-415e-b074-c2bc63bbdaa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c8ccb1d8a104314a3bdf30106c20ef7)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:46.109167 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb54d19f-d0fd-415e-b074-c2bc63bbdaa4 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:46.159952 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cb54d19f-d0fd-415e-b074-c2bc63bbdaa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c8ccb1d8a104314a3bdf30106c20ef7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:46.167291 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cb54d19f-d0fd-415e-b074-c2bc63bbdaa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c8ccb1d8a104314a3bdf30106c20ef7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:46.175272 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cb54d19f-d0fd-415e-b074-c2bc63bbdaa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c8ccb1d8a104314a3bdf30106c20ef7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:46.176092 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cb54d19f-d0fd-415e-b074-c2bc63bbdaa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c8ccb1d8a104314a3bdf30106c20ef7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:46.176713 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-cb54d19f-d0fd-415e-b074-c2bc63bbdaa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c8ccb1d8a104314a3bdf30106c20ef7'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:46.194981 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1359/2676] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:46 2022] DELETE /identity/v3/users/1c8ccb1d8a104314a3bdf30106c20ef7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:46.200050 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548263d2-f4e8-4006-8f25-63bcf0db6025 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:46.200595 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548263d2-f4e8-4006-8f25-63bcf0db6025 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:46.200595 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548263d2-f4e8-4006-8f25-63bcf0db6025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:46.220288 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-548263d2-f4e8-4006-8f25-63bcf0db6025 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:46.221635 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-548263d2-f4e8-4006-8f25-63bcf0db6025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:46.249198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54d19f-d0fd-415e-b074-c2bc63bbdaa4 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:46.250803 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-cb54d19f-d0fd-415e-b074-c2bc63bbdaa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:46.277438 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:46.281335 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:46.296652 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1318/2677] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:46.309466 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548263d2-f4e8-4006-8f25-63bcf0db6025 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:46.311635 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-548263d2-f4e8-4006-8f25-63bcf0db6025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:46.344774 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54d19f-d0fd-415e-b074-c2bc63bbdaa4 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:46.353483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02467bea-e969-448a-88cd-3a9dcc0f6e42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:46.354402 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02467bea-e969-448a-88cd-3a9dcc0f6e42 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:46.354821 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02467bea-e969-448a-88cd-3a9dcc0f6e42 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:46.355243 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02467bea-e969-448a-88cd-3a9dcc0f6e42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:46.357913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-02467bea-e969-448a-88cd-3a9dcc0f6e42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:46.380797 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548263d2-f4e8-4006-8f25-63bcf0db6025 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:46.384790 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf20477-3f0d-47b6-ac68-3405c8112c25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:46.385848 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf20477-3f0d-47b6-ac68-3405c8112c25 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:46.386138 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf20477-3f0d-47b6-ac68-3405c8112c25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:46.386424 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf20477-3f0d-47b6-ac68-3405c8112c25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:46.396325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf20477-3f0d-47b6-ac68-3405c8112c25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:46.397990 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf20477-3f0d-47b6-ac68-3405c8112c25 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:46.410342 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1319/2678] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:39:46.441839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02467bea-e969-448a-88cd-3a9dcc0f6e42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:46.444827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02467bea-e969-448a-88cd-3a9dcc0f6e42 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:46.462877 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1360/2679] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:39:46.778686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf20477-3f0d-47b6-ac68-3405c8112c25 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:46.780551 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4bf20477-3f0d-47b6-ac68-3405c8112c25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:46.797121 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf20477-3f0d-47b6-ac68-3405c8112c25 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:46.802004 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d415f7-8f66-4779-960f-4aa6fa9b64b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:46.803281 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d415f7-8f66-4779-960f-4aa6fa9b64b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:46.803634 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d415f7-8f66-4779-960f-4aa6fa9b64b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:46.804185 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d415f7-8f66-4779-960f-4aa6fa9b64b4 admin admin] PATH_INFO: `/v3/projects/44354b71c294460a9eac3b94241ffb55` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:46.806045 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d415f7-8f66-4779-960f-4aa6fa9b64b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44354b71c294460a9eac3b94241ffb55)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:46.808478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d415f7-8f66-4779-960f-4aa6fa9b64b4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:46.858672 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-24d415f7-8f66-4779-960f-4aa6fa9b64b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44354b71c294460a9eac3b94241ffb55', 'request_id': 'req-24d415f7-8f66-4779-960f-4aa6fa9b64b4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:46.864710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-24d415f7-8f66-4779-960f-4aa6fa9b64b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44354b71c294460a9eac3b94241ffb55', 'request_id': 'req-24d415f7-8f66-4779-960f-4aa6fa9b64b4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:46.873431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-24d415f7-8f66-4779-960f-4aa6fa9b64b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44354b71c294460a9eac3b94241ffb55', 'request_id': 'req-24d415f7-8f66-4779-960f-4aa6fa9b64b4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:46.886799 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-24d415f7-8f66-4779-960f-4aa6fa9b64b4 admin admin] The token cache is being invalidate because project 44354b71c294460a9eac3b94241ffb55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:39:46.887153 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-24d415f7-8f66-4779-960f-4aa6fa9b64b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:46.887867 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-24d415f7-8f66-4779-960f-4aa6fa9b64b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:46.889530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1320/2680] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:46 2022] DELETE /identity/v3/projects/44354b71c294460a9eac3b94241ffb55 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:46.894525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02467bea-e969-448a-88cd-3a9dcc0f6e42 service nova] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:46.907012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02467bea-e969-448a-88cd-3a9dcc0f6e42 service nova] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:46.979678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b598d837-881d-4b42-8b4b-53e3ce4b758a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:46.980379 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598d837-881d-4b42-8b4b-53e3ce4b758a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:46.980658 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598d837-881d-4b42-8b4b-53e3ce4b758a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:46.980935 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b598d837-881d-4b42-8b4b-53e3ce4b758a admin admin] PATH_INFO: `/v3/users/72bdf90ee068430dac22057c6f767fd3` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:46.982083 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b598d837-881d-4b42-8b4b-53e3ce4b758a admin admin] RBAC: Authorizing `identity:delete_user(user_id=72bdf90ee068430dac22057c6f767fd3)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:46.984250 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b598d837-881d-4b42-8b4b-53e3ce4b758a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:47.049530 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b598d837-881d-4b42-8b4b-53e3ce4b758a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72bdf90ee068430dac22057c6f767fd3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:47.059148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b598d837-881d-4b42-8b4b-53e3ce4b758a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72bdf90ee068430dac22057c6f767fd3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:47.069607 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b598d837-881d-4b42-8b4b-53e3ce4b758a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72bdf90ee068430dac22057c6f767fd3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:47.070617 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b598d837-881d-4b42-8b4b-53e3ce4b758a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72bdf90ee068430dac22057c6f767fd3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:47.071589 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b598d837-881d-4b42-8b4b-53e3ce4b758a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72bdf90ee068430dac22057c6f767fd3'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:47.073825 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:47.098779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1361/2681] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:46 2022] DELETE /identity/v3/users/72bdf90ee068430dac22057c6f767fd3 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:47.407609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d415f7-8f66-4779-960f-4aa6fa9b64b4 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:47.409283 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-24d415f7-8f66-4779-960f-4aa6fa9b64b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:47.429204 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d415f7-8f66-4779-960f-4aa6fa9b64b4 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:47.492029 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1b438-a4e6-4c0b-98e5-e083204a65e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:47.492715 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1b438-a4e6-4c0b-98e5-e083204a65e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:47.492978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1b438-a4e6-4c0b-98e5-e083204a65e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:47.493277 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1b438-a4e6-4c0b-98e5-e083204a65e5 admin admin] PATH_INFO: `/v3/projects/05e8fa04025d4558bacc99faf06f92dc` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:47.494767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1b438-a4e6-4c0b-98e5-e083204a65e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05e8fa04025d4558bacc99faf06f92dc)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:47.496383 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1b438-a4e6-4c0b-98e5-e083204a65e5 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:47.530221 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:47.555208 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f8c1b438-a4e6-4c0b-98e5-e083204a65e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05e8fa04025d4558bacc99faf06f92dc', 'request_id': 'req-f8c1b438-a4e6-4c0b-98e5-e083204a65e5'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:47.561985 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f8c1b438-a4e6-4c0b-98e5-e083204a65e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05e8fa04025d4558bacc99faf06f92dc', 'request_id': 'req-f8c1b438-a4e6-4c0b-98e5-e083204a65e5'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:47.570951 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f8c1b438-a4e6-4c0b-98e5-e083204a65e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05e8fa04025d4558bacc99faf06f92dc', 'request_id': 'req-f8c1b438-a4e6-4c0b-98e5-e083204a65e5'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:47.583772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f8c1b438-a4e6-4c0b-98e5-e083204a65e5 admin admin] The token cache is being invalidate because project 05e8fa04025d4558bacc99faf06f92dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:39:47.584008 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f8c1b438-a4e6-4c0b-98e5-e083204a65e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:47.584669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-f8c1b438-a4e6-4c0b-98e5-e083204a65e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:47.586716 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1321/2682] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:47 2022] DELETE /identity/v3/projects/05e8fa04025d4558bacc99faf06f92dc => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:39:47.591812 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81348caa-6fef-41b6-83c5-a70ee14b1f23 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:47.592322 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81348caa-6fef-41b6-83c5-a70ee14b1f23 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:47.592504 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81348caa-6fef-41b6-83c5-a70ee14b1f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:47.612573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-81348caa-6fef-41b6-83c5-a70ee14b1f23 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:47.613813 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-81348caa-6fef-41b6-83c5-a70ee14b1f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:47.674702 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1362/2683] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:47.683794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047386c-3e62-427a-b3fb-10e9625ebad7 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:47.684110 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047386c-3e62-427a-b3fb-10e9625ebad7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:47.684386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047386c-3e62-427a-b3fb-10e9625ebad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:47.705437 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-c047386c-3e62-427a-b3fb-10e9625ebad7 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:47.706642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c047386c-3e62-427a-b3fb-10e9625ebad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:47.714901 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1322/2684] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:47.718976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81348caa-6fef-41b6-83c5-a70ee14b1f23 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:47.728998 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81348caa-6fef-41b6-83c5-a70ee14b1f23 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:47.733762 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f1ee2-15f0-4df9-94c1-fb3544327764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:47.734376 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f1ee2-15f0-4df9-94c1-fb3544327764 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:47.734655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f1ee2-15f0-4df9-94c1-fb3544327764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:47.734927 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f1ee2-15f0-4df9-94c1-fb3544327764 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:47.735587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f1ee2-15f0-4df9-94c1-fb3544327764 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:39:47.735912 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f1ee2-15f0-4df9-94c1-fb3544327764 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:47.737505 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f1ee2-15f0-4df9-94c1-fb3544327764 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:47.751362 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1363/2685] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:39:47.756048 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047386c-3e62-427a-b3fb-10e9625ebad7 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:47.766604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047386c-3e62-427a-b3fb-10e9625ebad7 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:47.771404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c283057-8718-46ad-b7aa-8529d1c91b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:47.772140 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c283057-8718-46ad-b7aa-8529d1c91b02 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:47.772492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c283057-8718-46ad-b7aa-8529d1c91b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:47.772835 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c283057-8718-46ad-b7aa-8529d1c91b02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:47.773540 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c283057-8718-46ad-b7aa-8529d1c91b02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:47.775498 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c283057-8718-46ad-b7aa-8529d1c91b02 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:47.807478 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1323/2686] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:47 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:47.812132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f1ee2-15f0-4df9-94c1-fb3544327764 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:47.824770 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f1ee2-15f0-4df9-94c1-fb3544327764 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:47.892753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ed234-d61f-44ce-ace6-ea73b5530888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:47.893304 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ed234-d61f-44ce-ace6-ea73b5530888 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:47.893619 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ed234-d61f-44ce-ace6-ea73b5530888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:47.893863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ed234-d61f-44ce-ace6-ea73b5530888 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:47.894505 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ed234-d61f-44ce-ace6-ea73b5530888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:47.896485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ed234-d61f-44ce-ace6-ea73b5530888 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:47.920727 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1364/2687] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:47 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:47.925463 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c283057-8718-46ad-b7aa-8529d1c91b02 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:47.940349 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c283057-8718-46ad-b7aa-8529d1c91b02 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:47.947280 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba75ad-b55f-4bed-9337-481aef4c00e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:47.948112 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba75ad-b55f-4bed-9337-481aef4c00e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:47.948484 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba75ad-b55f-4bed-9337-481aef4c00e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:47.948872 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba75ad-b55f-4bed-9337-481aef4c00e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:47.949665 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba75ad-b55f-4bed-9337-481aef4c00e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:47.951792 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba75ad-b55f-4bed-9337-481aef4c00e4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:47.964778 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1324/2688] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:39:47.971078 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ed234-d61f-44ce-ace6-ea73b5530888 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:47.982453 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ed234-d61f-44ce-ace6-ea73b5530888 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:47.987329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90507b54-f64f-4776-ba76-1d702101e729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:47.987994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90507b54-f64f-4776-ba76-1d702101e729 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:47.988276 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90507b54-f64f-4776-ba76-1d702101e729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:47.988557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90507b54-f64f-4776-ba76-1d702101e729 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:47.989220 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90507b54-f64f-4776-ba76-1d702101e729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:47.990855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90507b54-f64f-4776-ba76-1d702101e729 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:48.003568 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1365/2689] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:39:48.009248 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba75ad-b55f-4bed-9337-481aef4c00e4 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:48.021932 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba75ad-b55f-4bed-9337-481aef4c00e4 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:48.030241 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442ae33-c3b2-40aa-b9b2-9915bbdeeaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:48.031088 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442ae33-c3b2-40aa-b9b2-9915bbdeeaad admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:48.031493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442ae33-c3b2-40aa-b9b2-9915bbdeeaad admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:48.031924 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442ae33-c3b2-40aa-b9b2-9915bbdeeaad admin admin] PATH_INFO: `/v3/projects/d5f82d33dcee45cab9e7833b434915b2/users/b7d03686f460498395383e876c469c11/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:48.049648 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442ae33-c3b2-40aa-b9b2-9915bbdeeaad admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5f82d33dcee45cab9e7833b434915b2, user_id=b7d03686f460498395383e876c469c11, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:48.052691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442ae33-c3b2-40aa-b9b2-9915bbdeeaad admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:48.070344 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1325/2690] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:48 2022] PUT /identity/v3/projects/d5f82d33dcee45cab9e7833b434915b2/users/b7d03686f460498395383e876c469c11/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:39:48.087385 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b40843-7822-457d-9b47-79037cd21dec None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:48.087803 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b40843-7822-457d-9b47-79037cd21dec None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:48.090419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b40843-7822-457d-9b47-79037cd21dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:48.124428 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-06b40843-7822-457d-9b47-79037cd21dec None None] MFA Rules not processed for user `b7d03686f460498395383e876c469c11`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:48.125827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-06b40843-7822-457d-9b47-79037cd21dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:48.165981 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:48.245821 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1366/2691] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:48.250662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442ae33-c3b2-40aa-b9b2-9915bbdeeaad admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:48.285703 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b40843-7822-457d-9b47-79037cd21dec None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:48.285703 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-06b40843-7822-457d-9b47-79037cd21dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:48.352226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442ae33-c3b2-40aa-b9b2-9915bbdeeaad admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:48.358169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2f165-aa40-4bf2-ae20-e98d273f7df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:48.359060 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2f165-aa40-4bf2-ae20-e98d273f7df6 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:48.359523 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2f165-aa40-4bf2-ae20-e98d273f7df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:48.359923 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2f165-aa40-4bf2-ae20-e98d273f7df6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:48.360778 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2f165-aa40-4bf2-ae20-e98d273f7df6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:48.363420 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2f165-aa40-4bf2-ae20-e98d273f7df6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:48.384131 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b40843-7822-457d-9b47-79037cd21dec None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:48.393868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd66de1-30c8-477a-a908-d06b731da5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:48.394873 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd66de1-30c8-477a-a908-d06b731da5e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:48.395246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd66de1-30c8-477a-a908-d06b731da5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:48.395520 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1326/2692] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:48 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:48.395711 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd66de1-30c8-477a-a908-d06b731da5e9 admin admin] PATH_INFO: `/v3/users/a004d6c41d1140819d1470f7035f9317` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:48.397568 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd66de1-30c8-477a-a908-d06b731da5e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a004d6c41d1140819d1470f7035f9317)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:48.402055 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2f165-aa40-4bf2-ae20-e98d273f7df6 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:48.402296 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd66de1-30c8-477a-a908-d06b731da5e9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:48.453666 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:48.474185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1dd66de1-30c8-477a-a908-d06b731da5e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a004d6c41d1140819d1470f7035f9317'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:48.482429 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2f165-aa40-4bf2-ae20-e98d273f7df6 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:48.485973 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1dd66de1-30c8-477a-a908-d06b731da5e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a004d6c41d1140819d1470f7035f9317'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:48.488627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a76dd-be2c-46f6-8536-caf7c77b1bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:48.489304 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a76dd-be2c-46f6-8536-caf7c77b1bcb admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:48.489767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a76dd-be2c-46f6-8536-caf7c77b1bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:48.490234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a76dd-be2c-46f6-8536-caf7c77b1bcb admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:48.491046 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a76dd-be2c-46f6-8536-caf7c77b1bcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:48.493809 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a76dd-be2c-46f6-8536-caf7c77b1bcb admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:48.497564 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1dd66de1-30c8-477a-a908-d06b731da5e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a004d6c41d1140819d1470f7035f9317'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:48.498548 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1dd66de1-30c8-477a-a908-d06b731da5e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a004d6c41d1140819d1470f7035f9317'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:48.499486 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-1dd66de1-30c8-477a-a908-d06b731da5e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a004d6c41d1140819d1470f7035f9317'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:48.522860 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1327/2693] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:48 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:39:48.525459 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a76dd-be2c-46f6-8536-caf7c77b1bcb admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:48.527817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1367/2694] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:48 2022] DELETE /identity/v3/users/a004d6c41d1140819d1470f7035f9317 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:48.530650 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1b6a76dd-be2c-46f6-8536-caf7c77b1bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:48.532438 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d25a0-878a-4c33-961c-5ba7aaaf90a3 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:48.532808 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d25a0-878a-4c33-961c-5ba7aaaf90a3 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:48.533252 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d25a0-878a-4c33-961c-5ba7aaaf90a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:48.549859 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a76dd-be2c-46f6-8536-caf7c77b1bcb admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:48.555613 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-c24d25a0-878a-4c33-961c-5ba7aaaf90a3 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:48.557111 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c24d25a0-878a-4c33-961c-5ba7aaaf90a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:48.611350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:48.629917 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1368/2695] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:39:48.633777 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c180f-7c31-4769-88c1-eb69b09f42cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:48.634677 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c180f-7c31-4769-88c1-eb69b09f42cf admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:48.635469 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c180f-7c31-4769-88c1-eb69b09f42cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:48.635594 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c180f-7c31-4769-88c1-eb69b09f42cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:48.636541 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c180f-7c31-4769-88c1-eb69b09f42cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:48.639167 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c180f-7c31-4769-88c1-eb69b09f42cf admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:48.643726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d25a0-878a-4c33-961c-5ba7aaaf90a3 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:48.645247 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c24d25a0-878a-4c33-961c-5ba7aaaf90a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:48.659396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1328/2696] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:39:48.661292 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c180f-7c31-4769-88c1-eb69b09f42cf admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:48.675711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c180f-7c31-4769-88c1-eb69b09f42cf admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:48.683279 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fdd47-f1e4-430d-a281-48cab5527643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:48.685317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fdd47-f1e4-430d-a281-48cab5527643 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:48.685317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fdd47-f1e4-430d-a281-48cab5527643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:48.685317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629fdd47-f1e4-430d-a281-48cab5527643 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:48.686301 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629fdd47-f1e4-430d-a281-48cab5527643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:48.688150 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629fdd47-f1e4-430d-a281-48cab5527643 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:48.704377 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1329/2697] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:39:48.709080 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fdd47-f1e4-430d-a281-48cab5527643 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:48.717010 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d25a0-878a-4c33-961c-5ba7aaaf90a3 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:48.722714 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b2a55-6b4c-4926-b889-838bd1608c3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:48.723492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629fdd47-f1e4-430d-a281-48cab5527643 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:48.723689 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b2a55-6b4c-4926-b889-838bd1608c3b service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:48.724097 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b2a55-6b4c-4926-b889-838bd1608c3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:48.724522 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b2a55-6b4c-4926-b889-838bd1608c3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:48.729866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588a06d-9d6d-4d07-9422-3b5b64efca75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:48.730619 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588a06d-9d6d-4d07-9422-3b5b64efca75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:48.730960 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588a06d-9d6d-4d07-9422-3b5b64efca75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:48.731296 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588a06d-9d6d-4d07-9422-3b5b64efca75 admin admin] PATH_INFO: `/v3/projects/e46c72fb4a09453f88aa81c923aa36d2/users/947db308121f4f4e92bcce33228cab43/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:48.739774 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7b2a55-6b4c-4926-b889-838bd1608c3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:48.742920 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7b2a55-6b4c-4926-b889-838bd1608c3b service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:48.749819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588a06d-9d6d-4d07-9422-3b5b64efca75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e46c72fb4a09453f88aa81c923aa36d2, user_id=947db308121f4f4e92bcce33228cab43, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:48.753091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588a06d-9d6d-4d07-9422-3b5b64efca75 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:48.756093 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1369/2698] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:39:48.771528 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1330/2699] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:48 2022] PUT /identity/v3/projects/e46c72fb4a09453f88aa81c923aa36d2/users/947db308121f4f4e92bcce33228cab43/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:39:48.786955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c70ae4-5130-4bb9-aeeb-b973d6c8b002 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:48.787392 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c70ae4-5130-4bb9-aeeb-b973d6c8b002 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:48.787901 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c70ae4-5130-4bb9-aeeb-b973d6c8b002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:48.824774 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f8c70ae4-5130-4bb9-aeeb-b973d6c8b002 None None] MFA Rules not processed for user `947db308121f4f4e92bcce33228cab43`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:48.826048 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f8c70ae4-5130-4bb9-aeeb-b973d6c8b002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:48.907193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1370/2700] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:39:48.912317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588a06d-9d6d-4d07-9422-3b5b64efca75 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:49.008604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588a06d-9d6d-4d07-9422-3b5b64efca75 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:49.014891 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25c49a-3010-463b-bb1d-d6eb7d345d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:49.015586 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25c49a-3010-463b-bb1d-d6eb7d345d1f admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:49.015990 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25c49a-3010-463b-bb1d-d6eb7d345d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:49.016436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25c49a-3010-463b-bb1d-d6eb7d345d1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:49.017436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c25c49a-3010-463b-bb1d-d6eb7d345d1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:49.019612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c25c49a-3010-463b-bb1d-d6eb7d345d1f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:49.055605 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1331/2701] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:48 2022] POST /identity/v3/projects => generated 375 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:39:49.065831 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c70ae4-5130-4bb9-aeeb-b973d6c8b002 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:49.084327 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c70ae4-5130-4bb9-aeeb-b973d6c8b002 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:49.144525 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25c49a-3010-463b-bb1d-d6eb7d345d1f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:49.146777 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0c25c49a-3010-463b-bb1d-d6eb7d345d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:49.184120 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:49.192752 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa5260-589e-414a-94fd-1a5389bbf04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:49.193434 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa5260-589e-414a-94fd-1a5389bbf04a admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:49.193976 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa5260-589e-414a-94fd-1a5389bbf04a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:49.194262 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa5260-589e-414a-94fd-1a5389bbf04a admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:49.195587 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa5260-589e-414a-94fd-1a5389bbf04a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:49.198779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa5260-589e-414a-94fd-1a5389bbf04a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:49.227085 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1371/2702] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:49 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:39:49.231132 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa5260-589e-414a-94fd-1a5389bbf04a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:49.248372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa5260-589e-414a-94fd-1a5389bbf04a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:49.255649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576d350-4e7a-44cd-92a9-a1e8fcfc848b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:49.256476 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576d350-4e7a-44cd-92a9-a1e8fcfc848b admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:49.256900 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576d350-4e7a-44cd-92a9-a1e8fcfc848b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:49.257382 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576d350-4e7a-44cd-92a9-a1e8fcfc848b admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:49.258483 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576d350-4e7a-44cd-92a9-a1e8fcfc848b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:49.261405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576d350-4e7a-44cd-92a9-a1e8fcfc848b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:49.262665 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25c49a-3010-463b-bb1d-d6eb7d345d1f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:49.276918 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edc721-553b-4a48-997d-c4c5444fbbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:49.278005 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edc721-553b-4a48-997d-c4c5444fbbf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:49.278790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1372/2703] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:39:49.282329 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edc721-553b-4a48-997d-c4c5444fbbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:49.282858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edc721-553b-4a48-997d-c4c5444fbbf3 admin admin] PATH_INFO: `/v3/projects/cabee0f9db4b41abba9fced3cf293eb9` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:49.288055 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67edc721-553b-4a48-997d-c4c5444fbbf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cabee0f9db4b41abba9fced3cf293eb9)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:49.288303 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576d350-4e7a-44cd-92a9-a1e8fcfc848b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:49.288617 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67edc721-553b-4a48-997d-c4c5444fbbf3 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:49.314880 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576d350-4e7a-44cd-92a9-a1e8fcfc848b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:49.322965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863aba3-0919-43aa-b133-cdf9ce1986b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:49.325069 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863aba3-0919-43aa-b133-cdf9ce1986b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:49.326194 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863aba3-0919-43aa-b133-cdf9ce1986b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:49.327067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863aba3-0919-43aa-b133-cdf9ce1986b1 admin admin] PATH_INFO: `/v3/projects/28717ddd51434aeab81dcf928d3e46be/users/6621e23a76a14fb19ec46c100c3c37c4/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:49.332061 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:49.354673 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863aba3-0919-43aa-b133-cdf9ce1986b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28717ddd51434aeab81dcf928d3e46be, user_id=6621e23a76a14fb19ec46c100c3c37c4, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:49.360520 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863aba3-0919-43aa-b133-cdf9ce1986b1 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:49.370076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-67edc721-553b-4a48-997d-c4c5444fbbf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cabee0f9db4b41abba9fced3cf293eb9', 'request_id': 'req-67edc721-553b-4a48-997d-c4c5444fbbf3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:49.378809 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-67edc721-553b-4a48-997d-c4c5444fbbf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cabee0f9db4b41abba9fced3cf293eb9', 'request_id': 'req-67edc721-553b-4a48-997d-c4c5444fbbf3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:49.390112 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-67edc721-553b-4a48-997d-c4c5444fbbf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cabee0f9db4b41abba9fced3cf293eb9', 'request_id': 'req-67edc721-553b-4a48-997d-c4c5444fbbf3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:49.390568 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1373/2704] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:49 2022] PUT /identity/v3/projects/28717ddd51434aeab81dcf928d3e46be/users/6621e23a76a14fb19ec46c100c3c37c4/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:39:49.396364 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863aba3-0919-43aa-b133-cdf9ce1986b1 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:49.411821 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-67edc721-553b-4a48-997d-c4c5444fbbf3 admin admin] The token cache is being invalidate because project cabee0f9db4b41abba9fced3cf293eb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:39:49.412564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-67edc721-553b-4a48-997d-c4c5444fbbf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:49.414948 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-67edc721-553b-4a48-997d-c4c5444fbbf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:49.417531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1332/2705] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:49 2022] DELETE /identity/v3/projects/cabee0f9db4b41abba9fced3cf293eb9 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:49.425562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe4274-4354-429a-849d-1ab27093d204 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:49.426196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe4274-4354-429a-849d-1ab27093d204 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:49.426698 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe4274-4354-429a-849d-1ab27093d204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:49.458056 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b6fe4274-4354-429a-849d-1ab27093d204 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:49.460408 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b6fe4274-4354-429a-849d-1ab27093d204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:49.502631 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863aba3-0919-43aa-b133-cdf9ce1986b1 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:49.509993 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8adadee-522d-4181-be18-7bd570c7f8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:49.510933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adadee-522d-4181-be18-7bd570c7f8ad admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:49.511419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adadee-522d-4181-be18-7bd570c7f8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:49.511897 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adadee-522d-4181-be18-7bd570c7f8ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:49.513093 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8adadee-522d-4181-be18-7bd570c7f8ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:49.516411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8adadee-522d-4181-be18-7bd570c7f8ad admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:49.538312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1374/2706] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:39:49.546568 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8adadee-522d-4181-be18-7bd570c7f8ad admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:49.549127 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e8adadee-522d-4181-be18-7bd570c7f8ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:49.561596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1333/2707] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:49.572649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d1a3b-1133-4697-b372-00b658158dc8 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:49.573190 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d1a3b-1133-4697-b372-00b658158dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:49.573470 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d1a3b-1133-4697-b372-00b658158dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:49.578326 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8adadee-522d-4181-be18-7bd570c7f8ad admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:49.585579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b73855-343a-406d-86df-ef5262679256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:49.587120 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b73855-343a-406d-86df-ef5262679256 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:49.587646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b73855-343a-406d-86df-ef5262679256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:49.588251 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b73855-343a-406d-86df-ef5262679256 admin admin] PATH_INFO: `/v3/projects/28717ddd51434aeab81dcf928d3e46be/users/6621e23a76a14fb19ec46c100c3c37c4/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:49.590309 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b73855-343a-406d-86df-ef5262679256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28717ddd51434aeab81dcf928d3e46be, user_id=6621e23a76a14fb19ec46c100c3c37c4, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:49.593067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b73855-343a-406d-86df-ef5262679256 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:49.604992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-247d1a3b-1133-4697-b372-00b658158dc8 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:49.605966 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-247d1a3b-1133-4697-b372-00b658158dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:49.607272 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1375/2708] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:49 2022] PUT /identity/v3/projects/28717ddd51434aeab81dcf928d3e46be/users/6621e23a76a14fb19ec46c100c3c37c4/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:39:49.622098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6963e-bb1b-4bba-becd-e0c1d512fab0 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:49.623179 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6963e-bb1b-4bba-becd-e0c1d512fab0 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:49.623179 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6963e-bb1b-4bba-becd-e0c1d512fab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:49.667505 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:49.670834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-50a6963e-bb1b-4bba-becd-e0c1d512fab0 None None] MFA Rules not processed for user `6621e23a76a14fb19ec46c100c3c37c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:49.673120 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-50a6963e-bb1b-4bba-becd-e0c1d512fab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:49.697716 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1334/2709] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:39:49.703154 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d1a3b-1133-4697-b372-00b658158dc8 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:49.712681 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d1a3b-1133-4697-b372-00b658158dc8 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:49.719183 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b1d39-53a6-421c-9dd2-2f494f0694f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:49.719802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b1d39-53a6-421c-9dd2-2f494f0694f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:49.720114 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b1d39-53a6-421c-9dd2-2f494f0694f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:49.720380 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b1d39-53a6-421c-9dd2-2f494f0694f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:49.721032 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b1d39-53a6-421c-9dd2-2f494f0694f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:39:49.721327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b1d39-53a6-421c-9dd2-2f494f0694f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:49.722946 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b1d39-53a6-421c-9dd2-2f494f0694f5 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:49.735627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1335/2710] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:39:49.741063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b1d39-53a6-421c-9dd2-2f494f0694f5 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:49.751691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b1d39-53a6-421c-9dd2-2f494f0694f5 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:49.757746 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63152de5-9cc9-446d-9d06-4c691d5f110a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:49.758647 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63152de5-9cc9-446d-9d06-4c691d5f110a admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:49.759031 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63152de5-9cc9-446d-9d06-4c691d5f110a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:49.759469 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63152de5-9cc9-446d-9d06-4c691d5f110a admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:49.760483 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63152de5-9cc9-446d-9d06-4c691d5f110a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:49.763607 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63152de5-9cc9-446d-9d06-4c691d5f110a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:49.778611 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1376/2711] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:39:49.794357 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6963e-bb1b-4bba-becd-e0c1d512fab0 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:49.796036 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-50a6963e-bb1b-4bba-becd-e0c1d512fab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:49.801108 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1336/2712] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:39:49 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:39:49.806237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63152de5-9cc9-446d-9d06-4c691d5f110a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:49.876043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6963e-bb1b-4bba-becd-e0c1d512fab0 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:49.876763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63152de5-9cc9-446d-9d06-4c691d5f110a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:49.879783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:49.884150 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421c42e-6f2f-4d00-89b8-565bf90acc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:49.885054 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421c42e-6f2f-4d00-89b8-565bf90acc39 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:49.885443 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421c42e-6f2f-4d00-89b8-565bf90acc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:49.885854 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421c42e-6f2f-4d00-89b8-565bf90acc39 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:49.886799 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421c42e-6f2f-4d00-89b8-565bf90acc39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:49.889980 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421c42e-6f2f-4d00-89b8-565bf90acc39 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:49.920869 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1337/2713] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:39:49 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:39:49.925536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421c42e-6f2f-4d00-89b8-565bf90acc39 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:49.937242 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421c42e-6f2f-4d00-89b8-565bf90acc39 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:49.942406 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c9aaa-6187-4cfd-89a2-7747be4c29e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:49.943028 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c9aaa-6187-4cfd-89a2-7747be4c29e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:49.943305 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c9aaa-6187-4cfd-89a2-7747be4c29e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:49.943583 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c9aaa-6187-4cfd-89a2-7747be4c29e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:49.944277 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c9aaa-6187-4cfd-89a2-7747be4c29e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:49.945877 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c9aaa-6187-4cfd-89a2-7747be4c29e6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:49.952603 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62412a-3986-47fa-af07-048e240a09d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:49.953510 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62412a-3986-47fa-af07-048e240a09d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:49.953934 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62412a-3986-47fa-af07-048e240a09d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:49.954350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62412a-3986-47fa-af07-048e240a09d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:49.956691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1338/2714] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:39:49.956908 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ce62412a-3986-47fa-af07-048e240a09d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:49.961325 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c9aaa-6187-4cfd-89a2-7747be4c29e6 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:49.978359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c9aaa-6187-4cfd-89a2-7747be4c29e6 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:49.984879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ca95c-7140-4d64-87fc-24945161cccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:49.985663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ca95c-7140-4d64-87fc-24945161cccf admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:49.986058 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ca95c-7140-4d64-87fc-24945161cccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:49.986631 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ca95c-7140-4d64-87fc-24945161cccf admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:49.987591 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ca95c-7140-4d64-87fc-24945161cccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:49.989936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ca95c-7140-4d64-87fc-24945161cccf admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:50.008336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1339/2715] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:39:50.014052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ca95c-7140-4d64-87fc-24945161cccf admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:50.024658 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62412a-3986-47fa-af07-048e240a09d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:50.026166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62412a-3986-47fa-af07-048e240a09d2 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:50.029367 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ca95c-7140-4d64-87fc-24945161cccf admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:50.036205 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb818851-4d2b-47f4-9f7b-57a1958bdb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:50.036922 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1377/2716] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:39:50.037163 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb818851-4d2b-47f4-9f7b-57a1958bdb71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:50.037576 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb818851-4d2b-47f4-9f7b-57a1958bdb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:50.037995 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb818851-4d2b-47f4-9f7b-57a1958bdb71 admin admin] PATH_INFO: `/v3/projects/19b1e493a4874189a1bfaa562b1ddeb8/users/289bf98447364a2ba441ca6cfa2fffe9/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:50.062961 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb818851-4d2b-47f4-9f7b-57a1958bdb71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b1e493a4874189a1bfaa562b1ddeb8, user_id=289bf98447364a2ba441ca6cfa2fffe9, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:50.067446 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb818851-4d2b-47f4-9f7b-57a1958bdb71 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:50.086472 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1340/2717] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:39:50 2022] PUT /identity/v3/projects/19b1e493a4874189a1bfaa562b1ddeb8/users/289bf98447364a2ba441ca6cfa2fffe9/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:50.099475 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba9514-1986-4af5-9383-4dc2a2f4c011 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:50.099893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba9514-1986-4af5-9383-4dc2a2f4c011 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:50.100324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba9514-1986-4af5-9383-4dc2a2f4c011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:50.138869 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-3dba9514-1986-4af5-9383-4dc2a2f4c011 None None] MFA Rules not processed for user `289bf98447364a2ba441ca6cfa2fffe9`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:50.142191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3dba9514-1986-4af5-9383-4dc2a2f4c011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:50.242674 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1378/2718] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:50.254571 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb818851-4d2b-47f4-9f7b-57a1958bdb71 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:50.261808 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-fb818851-4d2b-47f4-9f7b-57a1958bdb71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:50.361142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb818851-4d2b-47f4-9f7b-57a1958bdb71 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:50.372334 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd588861-cdaf-4d09-adfb-c157917c2ae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:50.373137 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd588861-cdaf-4d09-adfb-c157917c2ae7 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:50.373610 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd588861-cdaf-4d09-adfb-c157917c2ae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:50.374063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd588861-cdaf-4d09-adfb-c157917c2ae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:50.385131 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:50.389875 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd588861-cdaf-4d09-adfb-c157917c2ae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:50.392495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd588861-cdaf-4d09-adfb-c157917c2ae7 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:50.408474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1341/2719] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:51.677575 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba9514-1986-4af5-9383-4dc2a2f4c011 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:51.687877 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba9514-1986-4af5-9383-4dc2a2f4c011 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:51.691445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64896557-bdb7-4a4f-8d7b-1540eb05105b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:51.692074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64896557-bdb7-4a4f-8d7b-1540eb05105b service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:51.692353 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64896557-bdb7-4a4f-8d7b-1540eb05105b service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:51.692639 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64896557-bdb7-4a4f-8d7b-1540eb05105b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:51.694388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-64896557-bdb7-4a4f-8d7b-1540eb05105b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:51.749204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=100 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:51.762534 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64896557-bdb7-4a4f-8d7b-1540eb05105b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:51.764081 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64896557-bdb7-4a4f-8d7b-1540eb05105b service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:51.774448 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1379/2720] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:52.502415 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd588861-cdaf-4d09-adfb-c157917c2ae7 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:52.511925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd588861-cdaf-4d09-adfb-c157917c2ae7 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:52.515842 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf51b5d-ac81-4f16-b12b-0f67b39a2c1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:52.516488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf51b5d-ac81-4f16-b12b-0f67b39a2c1f service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:52.516809 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf51b5d-ac81-4f16-b12b-0f67b39a2c1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:52.517045 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf51b5d-ac81-4f16-b12b-0f67b39a2c1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:52.583486 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf51b5d-ac81-4f16-b12b-0f67b39a2c1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:52.584960 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf51b5d-ac81-4f16-b12b-0f67b39a2c1f service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:52.596166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1342/2721] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:58.732155 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64896557-bdb7-4a4f-8d7b-1540eb05105b service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:58.924024 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64896557-bdb7-4a4f-8d7b-1540eb05105b service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:58.928898 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c819756-406d-4ab6-9ed1-d3c9038df17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:58.929600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c819756-406d-4ab6-9ed1-d3c9038df17d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:58.929921 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c819756-406d-4ab6-9ed1-d3c9038df17d admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:58.930236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c819756-406d-4ab6-9ed1-d3c9038df17d admin admin] PATH_INFO: `/v3/users/b7d03686f460498395383e876c469c11` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:58.931449 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c819756-406d-4ab6-9ed1-d3c9038df17d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7d03686f460498395383e876c469c11)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:58.933514 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c819756-406d-4ab6-9ed1-d3c9038df17d admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:58.988817 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7c819756-406d-4ab6-9ed1-d3c9038df17d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7d03686f460498395383e876c469c11'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:58.996069 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7c819756-406d-4ab6-9ed1-d3c9038df17d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7d03686f460498395383e876c469c11'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:59.004691 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7c819756-406d-4ab6-9ed1-d3c9038df17d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7d03686f460498395383e876c469c11'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:59.005423 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7c819756-406d-4ab6-9ed1-d3c9038df17d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7d03686f460498395383e876c469c11'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:59.006156 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-7c819756-406d-4ab6-9ed1-d3c9038df17d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7d03686f460498395383e876c469c11'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:59.024992 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1380/2722] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:58 2022] DELETE /identity/v3/users/b7d03686f460498395383e876c469c11 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:59.029593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7268d5-96f9-408e-83d4-391fe6b7d5cf None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:59.030364 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7268d5-96f9-408e-83d4-391fe6b7d5cf None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:59.030364 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7268d5-96f9-408e-83d4-391fe6b7d5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:59.050537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-0f7268d5-96f9-408e-83d4-391fe6b7d5cf None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:59.051811 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0f7268d5-96f9-408e-83d4-391fe6b7d5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:59.122136 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1343/2723] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:59.133651 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c819756-406d-4ab6-9ed1-d3c9038df17d admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:59.135280 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7c819756-406d-4ab6-9ed1-d3c9038df17d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:59.203227 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c819756-406d-4ab6-9ed1-d3c9038df17d admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:59.207077 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6b77b-559c-4142-87a2-70498dc473db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:59.207694 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6b77b-559c-4142-87a2-70498dc473db service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:59.208014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6b77b-559c-4142-87a2-70498dc473db service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:59.208295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6b77b-559c-4142-87a2-70498dc473db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:59.218251 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6b77b-559c-4142-87a2-70498dc473db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:59.219688 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6b77b-559c-4142-87a2-70498dc473db service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:59.230432 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1381/2724] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:59.329237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7268d5-96f9-408e-83d4-391fe6b7d5cf None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:59.330906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0f7268d5-96f9-408e-83d4-391fe6b7d5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:59.345251 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7268d5-96f9-408e-83d4-391fe6b7d5cf None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:59.350140 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc9df5-b4b5-4e69-8acc-939e81370ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:59.350791 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc9df5-b4b5-4e69-8acc-939e81370ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:59.351053 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc9df5-b4b5-4e69-8acc-939e81370ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:59.351330 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc9df5-b4b5-4e69-8acc-939e81370ce1 admin admin] PATH_INFO: `/v3/users/289bf98447364a2ba441ca6cfa2fffe9` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:59.352514 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc9df5-b4b5-4e69-8acc-939e81370ce1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=289bf98447364a2ba441ca6cfa2fffe9)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:59.354449 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc9df5-b4b5-4e69-8acc-939e81370ce1 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:59.405153 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-24cc9df5-b4b5-4e69-8acc-939e81370ce1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '289bf98447364a2ba441ca6cfa2fffe9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:59.413220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-24cc9df5-b4b5-4e69-8acc-939e81370ce1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '289bf98447364a2ba441ca6cfa2fffe9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:59.422577 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-24cc9df5-b4b5-4e69-8acc-939e81370ce1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '289bf98447364a2ba441ca6cfa2fffe9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:59.423430 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-24cc9df5-b4b5-4e69-8acc-939e81370ce1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '289bf98447364a2ba441ca6cfa2fffe9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:59.441785 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-24cc9df5-b4b5-4e69-8acc-939e81370ce1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '289bf98447364a2ba441ca6cfa2fffe9'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:39:59.449988 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1344/2725] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:39:59 2022] DELETE /identity/v3/users/289bf98447364a2ba441ca6cfa2fffe9 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:39:59.455629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5b837-7d1d-4296-b999-99de12973e1d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:59.456053 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5b837-7d1d-4296-b999-99de12973e1d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:59.456566 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5b837-7d1d-4296-b999-99de12973e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:59.504182 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-e2b5b837-7d1d-4296-b999-99de12973e1d None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:39:59.506748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e2b5b837-7d1d-4296-b999-99de12973e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:59.517707 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:59.597274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1382/2726] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:39:59.651033 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc9df5-b4b5-4e69-8acc-939e81370ce1 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:59.653938 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-24cc9df5-b4b5-4e69-8acc-939e81370ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:59.664264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:39:59.821003 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc9df5-b4b5-4e69-8acc-939e81370ce1 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:59.836266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002562a-c968-4ab7-8c0b-bdc2c9289969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:59.840314 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002562a-c968-4ab7-8c0b-bdc2c9289969 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:59.840723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002562a-c968-4ab7-8c0b-bdc2c9289969 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:59.841832 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002562a-c968-4ab7-8c0b-bdc2c9289969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:59.857703 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002562a-c968-4ab7-8c0b-bdc2c9289969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:59.859711 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002562a-c968-4ab7-8c0b-bdc2c9289969 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:39:59.893418 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5b837-7d1d-4296-b999-99de12973e1d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:39:59.895567 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e2b5b837-7d1d-4296-b999-99de12973e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:39:59.901532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1345/2727] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:39:59.933428 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5b837-7d1d-4296-b999-99de12973e1d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:39:59.943597 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd99383-b0ef-48c4-b46e-5321e8357dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:39:59.944599 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd99383-b0ef-48c4-b46e-5321e8357dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:39:59.945646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd99383-b0ef-48c4-b46e-5321e8357dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:39:59.946138 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd99383-b0ef-48c4-b46e-5321e8357dec admin admin] PATH_INFO: `/v3/projects/d5f82d33dcee45cab9e7833b434915b2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:39:59.947774 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd99383-b0ef-48c4-b46e-5321e8357dec admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5f82d33dcee45cab9e7833b434915b2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:39:59.950432 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd99383-b0ef-48c4-b46e-5321e8357dec admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:00.033928 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ebd99383-b0ef-48c4-b46e-5321e8357dec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5f82d33dcee45cab9e7833b434915b2', 'request_id': 'req-ebd99383-b0ef-48c4-b46e-5321e8357dec'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:00.045324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ebd99383-b0ef-48c4-b46e-5321e8357dec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5f82d33dcee45cab9e7833b434915b2', 'request_id': 'req-ebd99383-b0ef-48c4-b46e-5321e8357dec'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:00.056341 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ebd99383-b0ef-48c4-b46e-5321e8357dec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5f82d33dcee45cab9e7833b434915b2', 'request_id': 'req-ebd99383-b0ef-48c4-b46e-5321e8357dec'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:00.085212 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ebd99383-b0ef-48c4-b46e-5321e8357dec admin admin] The token cache is being invalidate because project d5f82d33dcee45cab9e7833b434915b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:40:00.085736 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ebd99383-b0ef-48c4-b46e-5321e8357dec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:00.086818 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-ebd99383-b0ef-48c4-b46e-5321e8357dec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:00.094071 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1383/2728] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:39:59 2022] DELETE /identity/v3/projects/d5f82d33dcee45cab9e7833b434915b2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:40:00.095854 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002562a-c968-4ab7-8c0b-bdc2c9289969 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:00.107882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002562a-c968-4ab7-8c0b-bdc2c9289969 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:00.206613 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d633b5b-0300-4c5c-8f40-fbd6db9c1842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:00.207547 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d633b5b-0300-4c5c-8f40-fbd6db9c1842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:00.207992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d633b5b-0300-4c5c-8f40-fbd6db9c1842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:00.208413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d633b5b-0300-4c5c-8f40-fbd6db9c1842 admin admin] PATH_INFO: `/v3/users/947db308121f4f4e92bcce33228cab43` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:00.210608 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d633b5b-0300-4c5c-8f40-fbd6db9c1842 admin admin] RBAC: Authorizing `identity:delete_user(user_id=947db308121f4f4e92bcce33228cab43)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:00.213854 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d633b5b-0300-4c5c-8f40-fbd6db9c1842 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:00.317238 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4d633b5b-0300-4c5c-8f40-fbd6db9c1842 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '947db308121f4f4e92bcce33228cab43'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:00.328722 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4d633b5b-0300-4c5c-8f40-fbd6db9c1842 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '947db308121f4f4e92bcce33228cab43'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:00.340729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4d633b5b-0300-4c5c-8f40-fbd6db9c1842 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '947db308121f4f4e92bcce33228cab43'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:00.341675 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4d633b5b-0300-4c5c-8f40-fbd6db9c1842 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '947db308121f4f4e92bcce33228cab43'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:00.364788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-4d633b5b-0300-4c5c-8f40-fbd6db9c1842 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '947db308121f4f4e92bcce33228cab43'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:00.372696 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1346/2729] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:40:00 2022] DELETE /identity/v3/users/947db308121f4f4e92bcce33228cab43 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:40:00.456230 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd99383-b0ef-48c4-b46e-5321e8357dec admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:00.458508 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ebd99383-b0ef-48c4-b46e-5321e8357dec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:00.481897 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd99383-b0ef-48c4-b46e-5321e8357dec admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:00.582418 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89781e08-413a-49a3-8610-8e432263bbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:00.583126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89781e08-413a-49a3-8610-8e432263bbb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:00.583410 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89781e08-413a-49a3-8610-8e432263bbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:00.584284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89781e08-413a-49a3-8610-8e432263bbb0 admin admin] PATH_INFO: `/v3/projects/19b1e493a4874189a1bfaa562b1ddeb8` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:00.586183 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89781e08-413a-49a3-8610-8e432263bbb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19b1e493a4874189a1bfaa562b1ddeb8)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:00.588767 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89781e08-413a-49a3-8610-8e432263bbb0 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:00.641538 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:00.659395 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-89781e08-413a-49a3-8610-8e432263bbb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19b1e493a4874189a1bfaa562b1ddeb8', 'request_id': 'req-89781e08-413a-49a3-8610-8e432263bbb0'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:00.681162 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-89781e08-413a-49a3-8610-8e432263bbb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19b1e493a4874189a1bfaa562b1ddeb8', 'request_id': 'req-89781e08-413a-49a3-8610-8e432263bbb0'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:00.693033 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-89781e08-413a-49a3-8610-8e432263bbb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19b1e493a4874189a1bfaa562b1ddeb8', 'request_id': 'req-89781e08-413a-49a3-8610-8e432263bbb0'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:00.716729 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-89781e08-413a-49a3-8610-8e432263bbb0 admin admin] The token cache is being invalidate because project 19b1e493a4874189a1bfaa562b1ddeb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:40:00.717397 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-89781e08-413a-49a3-8610-8e432263bbb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:00.719043 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-89781e08-413a-49a3-8610-8e432263bbb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:00.721745 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1384/2730] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:40:00 2022] DELETE /identity/v3/projects/19b1e493a4874189a1bfaa562b1ddeb8 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:40:00.731990 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af86b1-23b1-4f49-b08f-d63f734fb2a1 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:00.731990 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af86b1-23b1-4f49-b08f-d63f734fb2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:00.732317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af86b1-23b1-4f49-b08f-d63f734fb2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:00.769800 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e9af86b1-23b1-4f49-b08f-d63f734fb2a1 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:00.772087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e9af86b1-23b1-4f49-b08f-d63f734fb2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:00.801225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:00.891542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89781e08-413a-49a3-8610-8e432263bbb0 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:00.893481 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-89781e08-413a-49a3-8610-8e432263bbb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:00.908044 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1347/2731] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:00.922984 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01827c-21ba-4b14-ac0a-6a2c6f0814bc None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:00.923958 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01827c-21ba-4b14-ac0a-6a2c6f0814bc None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:00.924433 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01827c-21ba-4b14-ac0a-6a2c6f0814bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:00.969340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-5f01827c-21ba-4b14-ac0a-6a2c6f0814bc None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:00.971797 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5f01827c-21ba-4b14-ac0a-6a2c6f0814bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:00.997738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1348/2732] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:40:01.001824 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01827c-21ba-4b14-ac0a-6a2c6f0814bc None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:01.010803 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89781e08-413a-49a3-8610-8e432263bbb0 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:01.016574 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33793465-7373-4d01-a8f5-9c1a911d5ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:01.017439 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33793465-7373-4d01-a8f5-9c1a911d5ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:01.017917 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33793465-7373-4d01-a8f5-9c1a911d5ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:01.018437 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33793465-7373-4d01-a8f5-9c1a911d5ad4 admin admin] PATH_INFO: `/v3/projects/e46c72fb4a09453f88aa81c923aa36d2` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:01.019894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33793465-7373-4d01-a8f5-9c1a911d5ad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e46c72fb4a09453f88aa81c923aa36d2)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:01.022872 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33793465-7373-4d01-a8f5-9c1a911d5ad4 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:01.023682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01827c-21ba-4b14-ac0a-6a2c6f0814bc None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:01.030835 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa1d07-3d55-4e45-b33e-521304445951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:01.031822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa1d07-3d55-4e45-b33e-521304445951 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:01.032412 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa1d07-3d55-4e45-b33e-521304445951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:01.032946 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa1d07-3d55-4e45-b33e-521304445951 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:01.034129 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa1d07-3d55-4e45-b33e-521304445951 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:40:01.034616 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa1d07-3d55-4e45-b33e-521304445951 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:01.037941 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aa1d07-3d55-4e45-b33e-521304445951 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:01.068883 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1349/2733] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 29 20:40:01.071203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa1d07-3d55-4e45-b33e-521304445951 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:01.101406 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa1d07-3d55-4e45-b33e-521304445951 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:01.107102 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-33793465-7373-4d01-a8f5-9c1a911d5ad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e46c72fb4a09453f88aa81c923aa36d2', 'request_id': 'req-33793465-7373-4d01-a8f5-9c1a911d5ad4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:01.119633 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-33793465-7373-4d01-a8f5-9c1a911d5ad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e46c72fb4a09453f88aa81c923aa36d2', 'request_id': 'req-33793465-7373-4d01-a8f5-9c1a911d5ad4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:01.132594 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-33793465-7373-4d01-a8f5-9c1a911d5ad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e46c72fb4a09453f88aa81c923aa36d2', 'request_id': 'req-33793465-7373-4d01-a8f5-9c1a911d5ad4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:01.149269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-33793465-7373-4d01-a8f5-9c1a911d5ad4 admin admin] The token cache is being invalidate because project e46c72fb4a09453f88aa81c923aa36d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:40:01.149700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-33793465-7373-4d01-a8f5-9c1a911d5ad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:01.150696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-33793465-7373-4d01-a8f5-9c1a911d5ad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:01.152593 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1385/2734] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:40:00 2022] DELETE /identity/v3/projects/e46c72fb4a09453f88aa81c923aa36d2 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:40:01.159178 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33793465-7373-4d01-a8f5-9c1a911d5ad4 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:01.163340 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-33793465-7373-4d01-a8f5-9c1a911d5ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:01.211367 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b2fd8-1835-45e1-a59f-94776269fad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:01.212326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b2fd8-1835-45e1-a59f-94776269fad6 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:01.212745 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b2fd8-1835-45e1-a59f-94776269fad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:01.213155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b2fd8-1835-45e1-a59f-94776269fad6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:01.214121 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b2fd8-1835-45e1-a59f-94776269fad6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:01.217203 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b2fd8-1835-45e1-a59f-94776269fad6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:01.260579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1350/2735] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:40:01 2022] POST /identity/v3/projects => generated 371 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:40:01.265765 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b2fd8-1835-45e1-a59f-94776269fad6 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:01.268434 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5b7b2fd8-1835-45e1-a59f-94776269fad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:01.283750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:01.299236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33793465-7373-4d01-a8f5-9c1a911d5ad4 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:01.371454 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b2fd8-1835-45e1-a59f-94776269fad6 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:01.377342 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44660b4-e944-4f44-9d7b-06d9ea4a4802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:01.378059 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44660b4-e944-4f44-9d7b-06d9ea4a4802 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:01.378451 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44660b4-e944-4f44-9d7b-06d9ea4a4802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:01.378827 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44660b4-e944-4f44-9d7b-06d9ea4a4802 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:01.379578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44660b4-e944-4f44-9d7b-06d9ea4a4802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:01.380291 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d829036-eb07-4337-b137-8acde4341b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:01.380934 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d829036-eb07-4337-b137-8acde4341b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:01.381228 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d829036-eb07-4337-b137-8acde4341b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:01.381509 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d829036-eb07-4337-b137-8acde4341b92 admin admin] PATH_INFO: `/v3/users/6621e23a76a14fb19ec46c100c3c37c4` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:01.381909 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44660b4-e944-4f44-9d7b-06d9ea4a4802 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:01.382766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d829036-eb07-4337-b137-8acde4341b92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6621e23a76a14fb19ec46c100c3c37c4)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:01.385224 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d829036-eb07-4337-b137-8acde4341b92 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:01.417776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1351/2736] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:40:01 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:40:01.417776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223b2bd-e18d-4841-aaac-2303da5e2a8c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:01.417776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223b2bd-e18d-4841-aaac-2303da5e2a8c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:01.417776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223b2bd-e18d-4841-aaac-2303da5e2a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:01.450087 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-b223b2bd-e18d-4841-aaac-2303da5e2a8c None None] MFA Rules not processed for user `61709ccc1e244185b252179451cd860f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:01.451976 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b223b2bd-e18d-4841-aaac-2303da5e2a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:01.462489 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5d829036-eb07-4337-b137-8acde4341b92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6621e23a76a14fb19ec46c100c3c37c4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:01.473977 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:01.478352 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5d829036-eb07-4337-b137-8acde4341b92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6621e23a76a14fb19ec46c100c3c37c4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:01.498487 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5d829036-eb07-4337-b137-8acde4341b92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6621e23a76a14fb19ec46c100c3c37c4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:01.499756 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5d829036-eb07-4337-b137-8acde4341b92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6621e23a76a14fb19ec46c100c3c37c4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:01.500791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-5d829036-eb07-4337-b137-8acde4341b92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6621e23a76a14fb19ec46c100c3c37c4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:01.544487 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1386/2737] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:40:01 2022] DELETE /identity/v3/users/6621e23a76a14fb19ec46c100c3c37c4 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:40:01.546011 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d829036-eb07-4337-b137-8acde4341b92 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:01.548075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5d829036-eb07-4337-b137-8acde4341b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:01.560418 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1352/2738] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 29 20:40:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:01.625181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223b2bd-e18d-4841-aaac-2303da5e2a8c None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:01.629188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-b223b2bd-e18d-4841-aaac-2303da5e2a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:01.702707 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d829036-eb07-4337-b137-8acde4341b92 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:01.711428 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f75fc-281c-4d44-955f-7e548dd2fa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:01.712657 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f75fc-281c-4d44-955f-7e548dd2fa98 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:01.713256 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f75fc-281c-4d44-955f-7e548dd2fa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:01.713995 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f75fc-281c-4d44-955f-7e548dd2fa98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:01.715491 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f75fc-281c-4d44-955f-7e548dd2fa98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:01.719077 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f75fc-281c-4d44-955f-7e548dd2fa98 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:01.739622 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1387/2739] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:40:01.745494 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f75fc-281c-4d44-955f-7e548dd2fa98 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:01.768723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f75fc-281c-4d44-955f-7e548dd2fa98 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:01.776050 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223b2bd-e18d-4841-aaac-2303da5e2a8c None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:01.781482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b57866-3954-4184-88df-cac21971251e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:01.782885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b57866-3954-4184-88df-cac21971251e admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:01.784716 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b57866-3954-4184-88df-cac21971251e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:01.785286 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b57866-3954-4184-88df-cac21971251e admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:01.787228 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b57866-3954-4184-88df-cac21971251e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:01.803796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b57866-3954-4184-88df-cac21971251e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:01.826583 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1388/2740] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:40:01.830830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b57866-3954-4184-88df-cac21971251e admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:01.847337 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b57866-3954-4184-88df-cac21971251e admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:01.856812 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ac544-7376-4d02-823f-56ca03753542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:01.858100 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ac544-7376-4d02-823f-56ca03753542 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:01.858531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ac544-7376-4d02-823f-56ca03753542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:01.859002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ac544-7376-4d02-823f-56ca03753542 admin admin] PATH_INFO: `/v3/projects/ef6822fbd190460cb05e41f971c509a6/users/9252d8aeddba44fdbef969cb0e4f7a64/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:01.862984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:01.889665 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ac544-7376-4d02-823f-56ca03753542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef6822fbd190460cb05e41f971c509a6, user_id=9252d8aeddba44fdbef969cb0e4f7a64, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:01.892808 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50841e75-3bea-41b2-b522-3dc37c9cad28 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599129b7d8ff47efb145787414ddb80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:01.893785 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50841e75-3bea-41b2-b522-3dc37c9cad28 service swift] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:01.894250 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50841e75-3bea-41b2-b522-3dc37c9cad28 service swift] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:01.894696 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50841e75-3bea-41b2-b522-3dc37c9cad28 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:01.894999 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ac544-7376-4d02-823f-56ca03753542 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:01.920445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1389/2741] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:40:01 2022] PUT /identity/v3/projects/ef6822fbd190460cb05e41f971c509a6/users/9252d8aeddba44fdbef969cb0e4f7a64/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:40:01.931250 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552db08-b708-4f46-9e03-e73ed852a8e9 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:01.931482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552db08-b708-4f46-9e03-e73ed852a8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:01.931760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552db08-b708-4f46-9e03-e73ed852a8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:01.970335 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-0552db08-b708-4f46-9e03-e73ed852a8e9 None None] MFA Rules not processed for user `9252d8aeddba44fdbef969cb0e4f7a64`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:01.971559 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0552db08-b708-4f46-9e03-e73ed852a8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:01.990276 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50841e75-3bea-41b2-b522-3dc37c9cad28 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:01.992134 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50841e75-3bea-41b2-b522-3dc37c9cad28 service swift] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:02.002669 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1353/2742] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 29 20:40:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:40:02.003526 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50841e75-3bea-41b2-b522-3dc37c9cad28 service swift] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:02.005379 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-50841e75-3bea-41b2-b522-3dc37c9cad28 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:02.022337 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50841e75-3bea-41b2-b522-3dc37c9cad28 service swift] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:02.028010 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=100 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:02.108880 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1390/2743] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:40:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:40:02.122075 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552db08-b708-4f46-9e03-e73ed852a8e9 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:02.124002 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0552db08-b708-4f46-9e03-e73ed852a8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:02.131576 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bf198-4e21-44de-b532-e342dd802405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:02.132533 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bf198-4e21-44de-b532-e342dd802405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:02.132826 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bf198-4e21-44de-b532-e342dd802405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:02.133111 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bf198-4e21-44de-b532-e342dd802405 admin admin] PATH_INFO: `/v3/projects/28717ddd51434aeab81dcf928d3e46be` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:02.134526 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6bf198-4e21-44de-b532-e342dd802405 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28717ddd51434aeab81dcf928d3e46be)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:02.141211 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6bf198-4e21-44de-b532-e342dd802405 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:02.217190 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fe6bf198-4e21-44de-b532-e342dd802405 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28717ddd51434aeab81dcf928d3e46be', 'request_id': 'req-fe6bf198-4e21-44de-b532-e342dd802405'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:02.223508 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552db08-b708-4f46-9e03-e73ed852a8e9 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:02.223866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fe6bf198-4e21-44de-b532-e342dd802405 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28717ddd51434aeab81dcf928d3e46be', 'request_id': 'req-fe6bf198-4e21-44de-b532-e342dd802405'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:02.235963 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fe6bf198-4e21-44de-b532-e342dd802405 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28717ddd51434aeab81dcf928d3e46be', 'request_id': 'req-fe6bf198-4e21-44de-b532-e342dd802405'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:02.255362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fe6bf198-4e21-44de-b532-e342dd802405 admin admin] The token cache is being invalidate because project 28717ddd51434aeab81dcf928d3e46be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:40:02.255801 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fe6bf198-4e21-44de-b532-e342dd802405 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:02.256705 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-fe6bf198-4e21-44de-b532-e342dd802405 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:02.261205 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1354/2744] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:40:02 2022] DELETE /identity/v3/projects/28717ddd51434aeab81dcf928d3e46be => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:40:02.268190 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00cf2c-2731-49a9-b61e-d49f74fa8b80 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:02.268550 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00cf2c-2731-49a9-b61e-d49f74fa8b80 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:02.268935 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00cf2c-2731-49a9-b61e-d49f74fa8b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:02.315703 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34036e-8a18-4e47-a5d9-d358b3f16a8a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d022ed8d2ac43ae816aa396a8e2f17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:02.316843 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34036e-8a18-4e47-a5d9-d358b3f16a8a service glance] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:02.317372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34036e-8a18-4e47-a5d9-d358b3f16a8a service glance] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:02.317803 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34036e-8a18-4e47-a5d9-d358b3f16a8a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:02.320692 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9c34036e-8a18-4e47-a5d9-d358b3f16a8a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:02.390295 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:02.429158 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34036e-8a18-4e47-a5d9-d358b3f16a8a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:02.435166 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34036e-8a18-4e47-a5d9-d358b3f16a8a service glance] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:02.466722 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1391/2745] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:40:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:40:02.492336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-7b00cf2c-2731-49a9-b61e-d49f74fa8b80 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:02.493823 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7b00cf2c-2731-49a9-b61e-d49f74fa8b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:02.616802 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1355/2746] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:02.621975 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b4e80-845a-4ee6-a64e-4ce312812462 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:02.622260 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b4e80-845a-4ee6-a64e-4ce312812462 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:02.622536 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b4e80-845a-4ee6-a64e-4ce312812462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:02.646693 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a49b4e80-845a-4ee6-a64e-4ce312812462 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:02.649865 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a49b4e80-845a-4ee6-a64e-4ce312812462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:02.656372 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1392/2747] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:02.663661 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00cf2c-2731-49a9-b61e-d49f74fa8b80 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:02.690537 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00cf2c-2731-49a9-b61e-d49f74fa8b80 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:02.696110 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28032d9a-9470-4e27-a1dd-5360bfe97fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:02.696800 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28032d9a-9470-4e27-a1dd-5360bfe97fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:02.697388 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28032d9a-9470-4e27-a1dd-5360bfe97fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:02.697738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28032d9a-9470-4e27-a1dd-5360bfe97fe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:02.698570 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28032d9a-9470-4e27-a1dd-5360bfe97fe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81397) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:40:02.698951 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28032d9a-9470-4e27-a1dd-5360bfe97fe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:02.700662 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28032d9a-9470-4e27-a1dd-5360bfe97fe8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:02.721940 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1356/2748] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:40:02.726882 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b4e80-845a-4ee6-a64e-4ce312812462 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:02.739773 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b4e80-845a-4ee6-a64e-4ce312812462 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:02.756395 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ca6da-f38a-4b8c-a090-cf297fe5f908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:02.758163 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ca6da-f38a-4b8c-a090-cf297fe5f908 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:02.758653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ca6da-f38a-4b8c-a090-cf297fe5f908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:02.759150 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ca6da-f38a-4b8c-a090-cf297fe5f908 admin admin] PATH_INFO: `/v3/projects` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:02.760334 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ca6da-f38a-4b8c-a090-cf297fe5f908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:02.763012 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ca6da-f38a-4b8c-a090-cf297fe5f908 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:02.798439 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1393/2749] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:40:02 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:40:02.804513 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28032d9a-9470-4e27-a1dd-5360bfe97fe8 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:02.818975 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28032d9a-9470-4e27-a1dd-5360bfe97fe8 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:02.853448 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:02.922561 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a21ff-018d-43b9-86c4-d9fde8150516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:02.923382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a21ff-018d-43b9-86c4-d9fde8150516 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:02.923762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a21ff-018d-43b9-86c4-d9fde8150516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:02.924319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a21ff-018d-43b9-86c4-d9fde8150516 admin admin] PATH_INFO: `/v3/users` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:02.925042 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a21ff-018d-43b9-86c4-d9fde8150516 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:02.927089 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a21ff-018d-43b9-86c4-d9fde8150516 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:02.955598 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1357/2750] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:40:02 2022] POST /identity/v3/users => generated 292 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:40:02.960089 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ca6da-f38a-4b8c-a090-cf297fe5f908 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:02.973536 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ca6da-f38a-4b8c-a090-cf297fe5f908 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:02.979368 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37fec9-c7e3-4cfa-a474-86faf87ea588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:02.980261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37fec9-c7e3-4cfa-a474-86faf87ea588 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:02.980553 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37fec9-c7e3-4cfa-a474-86faf87ea588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:02.980808 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37fec9-c7e3-4cfa-a474-86faf87ea588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:02.981445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad37fec9-c7e3-4cfa-a474-86faf87ea588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:02.983021 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad37fec9-c7e3-4cfa-a474-86faf87ea588 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:02.996712 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1394/2751] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:40:02.999730 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a21ff-018d-43b9-86c4-d9fde8150516 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:03.017359 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a21ff-018d-43b9-86c4-d9fde8150516 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:03.024879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7219b4-d529-48bd-a674-04072efad5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:03.025853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7219b4-d529-48bd-a674-04072efad5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:03.026135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7219b4-d529-48bd-a674-04072efad5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:03.026389 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7219b4-d529-48bd-a674-04072efad5f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:03.027033 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7219b4-d529-48bd-a674-04072efad5f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:03.029058 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7219b4-d529-48bd-a674-04072efad5f7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:03.042122 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1358/2752] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:40:03.046737 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37fec9-c7e3-4cfa-a474-86faf87ea588 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:03.081197 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37fec9-c7e3-4cfa-a474-86faf87ea588 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:03.095775 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a6ffd-0300-4a61-9a73-34f2ad016a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:03.097334 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a6ffd-0300-4a61-9a73-34f2ad016a21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:03.097621 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a6ffd-0300-4a61-9a73-34f2ad016a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:03.098469 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a6ffd-0300-4a61-9a73-34f2ad016a21 admin admin] PATH_INFO: `/v3/projects/892e39233ce148dfb1b24f21737200ba/users/88320025d7114f85872d8427ef0c9cd3/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:03.129887 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583a6ffd-0300-4a61-9a73-34f2ad016a21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=892e39233ce148dfb1b24f21737200ba, user_id=88320025d7114f85872d8427ef0c9cd3, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:03.138102 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583a6ffd-0300-4a61-9a73-34f2ad016a21 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:03.158179 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1395/2753] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:40:03 2022] PUT /identity/v3/projects/892e39233ce148dfb1b24f21737200ba/users/88320025d7114f85872d8427ef0c9cd3/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:40:03.178502 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7d23f-e40b-48e1-aeef-705383db66fe None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:03.179963 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7d23f-e40b-48e1-aeef-705383db66fe None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:03.179963 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7d23f-e40b-48e1-aeef-705383db66fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:03.216237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-09e7d23f-e40b-48e1-aeef-705383db66fe None None] MFA Rules not processed for user `88320025d7114f85872d8427ef0c9cd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:03.218315 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-09e7d23f-e40b-48e1-aeef-705383db66fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:03.311261 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1359/2754] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:40:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:03.326269 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a6ffd-0300-4a61-9a73-34f2ad016a21 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:03.328187 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-583a6ffd-0300-4a61-9a73-34f2ad016a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:03.404408 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a6ffd-0300-4a61-9a73-34f2ad016a21 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:03.408446 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46adf755-f5f7-4fb1-b91c-63501664028b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:03.409138 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adf755-f5f7-4fb1-b91c-63501664028b service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:03.409575 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adf755-f5f7-4fb1-b91c-63501664028b service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:03.409911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adf755-f5f7-4fb1-b91c-63501664028b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:03.420447 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46adf755-f5f7-4fb1-b91c-63501664028b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:03.422672 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46adf755-f5f7-4fb1-b91c-63501664028b service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:03.433022 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1396/2755] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:40:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:40:06.317415 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7d23f-e40b-48e1-aeef-705383db66fe None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:06.453093 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7d23f-e40b-48e1-aeef-705383db66fe None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:06.458936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b350a7-5f6a-4085-8680-8cda9fe060ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:06.459675 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b350a7-5f6a-4085-8680-8cda9fe060ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:06.460602 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b350a7-5f6a-4085-8680-8cda9fe060ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:06.460947 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b350a7-5f6a-4085-8680-8cda9fe060ed admin admin] PATH_INFO: `/v3/users/88320025d7114f85872d8427ef0c9cd3` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:06.464485 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b350a7-5f6a-4085-8680-8cda9fe060ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=88320025d7114f85872d8427ef0c9cd3)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:06.466145 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b350a7-5f6a-4085-8680-8cda9fe060ed admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:06.513965 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:06.532038 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-26b350a7-5f6a-4085-8680-8cda9fe060ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88320025d7114f85872d8427ef0c9cd3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:06.544230 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-26b350a7-5f6a-4085-8680-8cda9fe060ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88320025d7114f85872d8427ef0c9cd3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:06.555340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-26b350a7-5f6a-4085-8680-8cda9fe060ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88320025d7114f85872d8427ef0c9cd3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:06.556238 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-26b350a7-5f6a-4085-8680-8cda9fe060ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88320025d7114f85872d8427ef0c9cd3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:06.571694 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-26b350a7-5f6a-4085-8680-8cda9fe060ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88320025d7114f85872d8427ef0c9cd3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:06.578469 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1360/2756] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:40:06 2022] DELETE /identity/v3/users/88320025d7114f85872d8427ef0c9cd3 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:40:06.583136 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4bca1c-b7a5-4563-a829-83033006106f None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:06.583624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4bca1c-b7a5-4563-a829-83033006106f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:06.583748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4bca1c-b7a5-4563-a829-83033006106f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:06.607277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4e4bca1c-b7a5-4563-a829-83033006106f None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:06.608629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4e4bca1c-b7a5-4563-a829-83033006106f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:06.699794 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1397/2757] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:06.709231 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b350a7-5f6a-4085-8680-8cda9fe060ed admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:06.711071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-26b350a7-5f6a-4085-8680-8cda9fe060ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:06.778508 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b350a7-5f6a-4085-8680-8cda9fe060ed admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:06.783188 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90dcf39-9b98-463e-a699-92b40e7fd35d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:06.783811 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dcf39-9b98-463e-a699-92b40e7fd35d service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:06.784125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dcf39-9b98-463e-a699-92b40e7fd35d service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:06.784573 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90dcf39-9b98-463e-a699-92b40e7fd35d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:06.794564 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90dcf39-9b98-463e-a699-92b40e7fd35d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:06.796040 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90dcf39-9b98-463e-a699-92b40e7fd35d service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:06.806873 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1361/2758] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:40:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:40:07.172796 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4bca1c-b7a5-4563-a829-83033006106f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:07.175577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4e4bca1c-b7a5-4563-a829-83033006106f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:07.197057 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4bca1c-b7a5-4563-a829-83033006106f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:07.207656 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d95e48-11d6-4641-b339-8245d065612c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:07.209939 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d95e48-11d6-4641-b339-8245d065612c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:07.210415 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d95e48-11d6-4641-b339-8245d065612c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:07.210878 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d95e48-11d6-4641-b339-8245d065612c admin admin] PATH_INFO: `/v3/projects/892e39233ce148dfb1b24f21737200ba` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:07.212547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d95e48-11d6-4641-b339-8245d065612c admin admin] RBAC: Authorizing `identity:delete_project(project_id=892e39233ce148dfb1b24f21737200ba)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:07.214582 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d95e48-11d6-4641-b339-8245d065612c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:07.239663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:07.287556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-05d95e48-11d6-4641-b339-8245d065612c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '892e39233ce148dfb1b24f21737200ba', 'request_id': 'req-05d95e48-11d6-4641-b339-8245d065612c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:07.298294 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-05d95e48-11d6-4641-b339-8245d065612c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '892e39233ce148dfb1b24f21737200ba', 'request_id': 'req-05d95e48-11d6-4641-b339-8245d065612c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:07.312685 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-05d95e48-11d6-4641-b339-8245d065612c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '892e39233ce148dfb1b24f21737200ba', 'request_id': 'req-05d95e48-11d6-4641-b339-8245d065612c'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:07.327157 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-05d95e48-11d6-4641-b339-8245d065612c admin admin] The token cache is being invalidate because project 892e39233ce148dfb1b24f21737200ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:40:07.327517 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-05d95e48-11d6-4641-b339-8245d065612c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:07.328161 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-05d95e48-11d6-4641-b339-8245d065612c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:07.329878 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1398/2759] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:40:07 2022] DELETE /identity/v3/projects/892e39233ce148dfb1b24f21737200ba => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:40:16.802836 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=41,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:17.048562 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b899012-fa2d-49a9-a3c2-9311098c97c3 None None] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:17.048845 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b899012-fa2d-49a9-a3c2-9311098c97c3 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:17.049208 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b899012-fa2d-49a9-a3c2-9311098c97c3 None None] PATH_INFO: `/` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:17.050441 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1362/2760] 10.222.0.18 () {54 vars in 921 bytes} [Mon Aug 29 20:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 29 20:40:17.055634 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108cb05-2aa9-4071-aa87-c9d7668e3d63 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:17.056169 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108cb05-2aa9-4071-aa87-c9d7668e3d63 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:17.056256 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108cb05-2aa9-4071-aa87-c9d7668e3d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:17.077540 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-2108cb05-2aa9-4071-aa87-c9d7668e3d63 None None] MFA Rules not processed for user `c0c44754a1c04a2b87081aba6cb509d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:17.078948 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2108cb05-2aa9-4071-aa87-c9d7668e3d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:17.153919 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1399/2761] 10.222.0.18 () {58 vars in 1018 bytes} [Mon Aug 29 20:40:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:17.166437 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b899012-fa2d-49a9-a3c2-9311098c97c3 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:17.168261 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9b899012-fa2d-49a9-a3c2-9311098c97c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:17.226592 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b899012-fa2d-49a9-a3c2-9311098c97c3 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:17.230165 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f943c6f-1d80-453e-b745-973a96f459c3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d022ed8d2ac43ae816aa396a8e2f17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:17.230763 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f943c6f-1d80-453e-b745-973a96f459c3 service glance] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:17.231040 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f943c6f-1d80-453e-b745-973a96f459c3 service glance] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:17.231309 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f943c6f-1d80-453e-b745-973a96f459c3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:17.241132 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f943c6f-1d80-453e-b745-973a96f459c3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:17.242534 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f943c6f-1d80-453e-b745-973a96f459c3 service glance] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:17.252959 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1363/2762] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:40:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:40:19.147585 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcf255-a7d5-42e5-889c-66835cb2f461 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:19.148130 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcf255-a7d5-42e5-889c-66835cb2f461 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:19.148239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcf255-a7d5-42e5-889c-66835cb2f461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:19.168098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-3adcf255-a7d5-42e5-889c-66835cb2f461 None None] MFA Rules not processed for user `61709ccc1e244185b252179451cd860f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:19.169355 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3adcf255-a7d5-42e5-889c-66835cb2f461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:19.226388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1400/2763] 10.222.0.18 () {58 vars in 1009 bytes} [Mon Aug 29 20:40:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:19.238272 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f943c6f-1d80-453e-b745-973a96f459c3 service glance] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:19.239690 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6f943c6f-1d80-453e-b745-973a96f459c3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:19.309065 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f943c6f-1d80-453e-b745-973a96f459c3 service glance] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:19.314216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25a776-a0b4-4a5c-adfe-f6a95bddc485 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599129b7d8ff47efb145787414ddb80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:19.314840 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25a776-a0b4-4a5c-adfe-f6a95bddc485 service swift] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:19.315112 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25a776-a0b4-4a5c-adfe-f6a95bddc485 service swift] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:19.315374 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25a776-a0b4-4a5c-adfe-f6a95bddc485 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:19.324350 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25a776-a0b4-4a5c-adfe-f6a95bddc485 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:19.326039 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d25a776-a0b4-4a5c-adfe-f6a95bddc485 service swift] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:19.334103 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1364/2764] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 29 20:40:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:40:29.219614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:29.330933 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=58 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:33.035788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcf255-a7d5-42e5-889c-66835cb2f461 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:33.037275 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3adcf255-a7d5-42e5-889c-66835cb2f461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:33.102632 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcf255-a7d5-42e5-889c-66835cb2f461 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:33.107014 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfd896-2dcd-4603-90ec-13c62b18acaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:33.107645 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfd896-2dcd-4603-90ec-13c62b18acaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:33.107927 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfd896-2dcd-4603-90ec-13c62b18acaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:33.108229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfd896-2dcd-4603-90ec-13c62b18acaa admin admin] PATH_INFO: `/v3/users/9252d8aeddba44fdbef969cb0e4f7a64` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:33.109369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfd896-2dcd-4603-90ec-13c62b18acaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=9252d8aeddba44fdbef969cb0e4f7a64)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:33.111277 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfd896-2dcd-4603-90ec-13c62b18acaa admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:33.158341 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-79dfd896-2dcd-4603-90ec-13c62b18acaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9252d8aeddba44fdbef969cb0e4f7a64'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:33.166480 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-79dfd896-2dcd-4603-90ec-13c62b18acaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9252d8aeddba44fdbef969cb0e4f7a64'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:33.173963 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-79dfd896-2dcd-4603-90ec-13c62b18acaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9252d8aeddba44fdbef969cb0e4f7a64'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:33.174611 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-79dfd896-2dcd-4603-90ec-13c62b18acaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9252d8aeddba44fdbef969cb0e4f7a64'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:33.175323 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-79dfd896-2dcd-4603-90ec-13c62b18acaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9252d8aeddba44fdbef969cb0e4f7a64'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:33.193366 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1401/2765] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:40:33 2022] DELETE /identity/v3/users/9252d8aeddba44fdbef969cb0e4f7a64 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:40:33.197900 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1da40-ec93-4fb2-b453-a779ee08490d None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:33.198369 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1da40-ec93-4fb2-b453-a779ee08490d None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:33.198613 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1da40-ec93-4fb2-b453-a779ee08490d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:33.217255 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e3d1da40-ec93-4fb2-b453-a779ee08490d None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:33.218457 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e3d1da40-ec93-4fb2-b453-a779ee08490d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:33.277443 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1365/2766] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:33.287683 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfd896-2dcd-4603-90ec-13c62b18acaa admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:33.289671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-79dfd896-2dcd-4603-90ec-13c62b18acaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:33.353648 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfd896-2dcd-4603-90ec-13c62b18acaa admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:33.357358 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8ac69-87b8-4173-932c-93a110a368ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:33.357972 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8ac69-87b8-4173-932c-93a110a368ad service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:33.358256 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8ac69-87b8-4173-932c-93a110a368ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:33.358553 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8ac69-87b8-4173-932c-93a110a368ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:33.367795 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a8ac69-87b8-4173-932c-93a110a368ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:33.369379 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a8ac69-87b8-4173-932c-93a110a368ad service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:33.379211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1402/2767] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:40:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:40:33.671375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1da40-ec93-4fb2-b453-a779ee08490d None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:33.672910 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e3d1da40-ec93-4fb2-b453-a779ee08490d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:33.686643 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1da40-ec93-4fb2-b453-a779ee08490d None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:33.692675 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd4950-b95f-4b3f-a0c2-f84ece898574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:33.693373 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd4950-b95f-4b3f-a0c2-f84ece898574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:33.693660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd4950-b95f-4b3f-a0c2-f84ece898574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:33.693942 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd4950-b95f-4b3f-a0c2-f84ece898574 admin admin] PATH_INFO: `/v3/projects/ef6822fbd190460cb05e41f971c509a6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:33.695354 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd4950-b95f-4b3f-a0c2-f84ece898574 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef6822fbd190460cb05e41f971c509a6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:33.697532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd4950-b95f-4b3f-a0c2-f84ece898574 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:33.764024 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e8cd4950-b95f-4b3f-a0c2-f84ece898574 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef6822fbd190460cb05e41f971c509a6', 'request_id': 'req-e8cd4950-b95f-4b3f-a0c2-f84ece898574'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:33.774673 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e8cd4950-b95f-4b3f-a0c2-f84ece898574 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef6822fbd190460cb05e41f971c509a6', 'request_id': 'req-e8cd4950-b95f-4b3f-a0c2-f84ece898574'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:33.792878 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e8cd4950-b95f-4b3f-a0c2-f84ece898574 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef6822fbd190460cb05e41f971c509a6', 'request_id': 'req-e8cd4950-b95f-4b3f-a0c2-f84ece898574'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:33.821261 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e8cd4950-b95f-4b3f-a0c2-f84ece898574 admin admin] The token cache is being invalidate because project ef6822fbd190460cb05e41f971c509a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:40:33.821935 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e8cd4950-b95f-4b3f-a0c2-f84ece898574 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:33.823280 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e8cd4950-b95f-4b3f-a0c2-f84ece898574 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:33.827658 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1366/2768] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:40:33 2022] DELETE /identity/v3/projects/ef6822fbd190460cb05e41f971c509a6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:40:33.833296 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c74673-08e2-401f-9f03-df66f096e37c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:33.833671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c74673-08e2-401f-9f03-df66f096e37c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:33.833984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c74673-08e2-401f-9f03-df66f096e37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:33.856215 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-64c74673-08e2-401f-9f03-df66f096e37c None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:33.858343 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-64c74673-08e2-401f-9f03-df66f096e37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:33.894651 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:33.948495 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1403/2769] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:33.957441 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead41a34-2a97-40a0-a698-8d82f20d9e3b None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:33.957724 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead41a34-2a97-40a0-a698-8d82f20d9e3b None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:33.958009 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead41a34-2a97-40a0-a698-8d82f20d9e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:33.986935 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-ead41a34-2a97-40a0-a698-8d82f20d9e3b None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:33.988243 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-ead41a34-2a97-40a0-a698-8d82f20d9e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:33.996163 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1367/2770] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:40:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:34.000600 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c74673-08e2-401f-9f03-df66f096e37c None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:34.010899 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c74673-08e2-401f-9f03-df66f096e37c None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:34.015741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a8330-82cd-494a-b877-cae1975e33ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:34.016392 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a8330-82cd-494a-b877-cae1975e33ec admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:34.016719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a8330-82cd-494a-b877-cae1975e33ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:34.016961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a8330-82cd-494a-b877-cae1975e33ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:34.017595 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46a8330-82cd-494a-b877-cae1975e33ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:40:34.017905 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46a8330-82cd-494a-b877-cae1975e33ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:34.019601 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46a8330-82cd-494a-b877-cae1975e33ec admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:34.040271 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1404/2771] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:40:34.041299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead41a34-2a97-40a0-a698-8d82f20d9e3b None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:34.052645 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead41a34-2a97-40a0-a698-8d82f20d9e3b None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:34.057922 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fb5aa-6b39-4d0e-9fcf-d049e13f7381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:34.058653 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fb5aa-6b39-4d0e-9fcf-d049e13f7381 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:34.059035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fb5aa-6b39-4d0e-9fcf-d049e13f7381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:34.059374 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708fb5aa-6b39-4d0e-9fcf-d049e13f7381 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:34.060166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708fb5aa-6b39-4d0e-9fcf-d049e13f7381 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:34.062556 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708fb5aa-6b39-4d0e-9fcf-d049e13f7381 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:34.103006 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1368/2772] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:40:34 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:40:34.107519 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a8330-82cd-494a-b877-cae1975e33ec admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:34.119868 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a8330-82cd-494a-b877-cae1975e33ec admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:34.179248 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c44339-8620-4633-a1e0-ac06d33b88a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:34.180418 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c44339-8620-4633-a1e0-ac06d33b88a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:34.180691 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c44339-8620-4633-a1e0-ac06d33b88a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:34.181003 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c44339-8620-4633-a1e0-ac06d33b88a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:34.181641 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c44339-8620-4633-a1e0-ac06d33b88a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:34.183580 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c44339-8620-4633-a1e0-ac06d33b88a5 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:34.207104 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1405/2773] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:40:34 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:40:34.212577 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fb5aa-6b39-4d0e-9fcf-d049e13f7381 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:34.222388 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708fb5aa-6b39-4d0e-9fcf-d049e13f7381 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:34.227175 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e929a2-6e1d-41d4-9ed2-a61405960695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:34.227788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e929a2-6e1d-41d4-9ed2-a61405960695 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:34.228106 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e929a2-6e1d-41d4-9ed2-a61405960695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:34.228380 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e929a2-6e1d-41d4-9ed2-a61405960695 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:34.229026 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e929a2-6e1d-41d4-9ed2-a61405960695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:34.230621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e929a2-6e1d-41d4-9ed2-a61405960695 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:34.241008 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1369/2774] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:40:34.245466 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c44339-8620-4633-a1e0-ac06d33b88a5 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:34.256025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c44339-8620-4633-a1e0-ac06d33b88a5 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:34.261417 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4ab21-e5e5-485c-9a80-8db31750cb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:34.262035 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4ab21-e5e5-485c-9a80-8db31750cb21 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:34.262321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4ab21-e5e5-485c-9a80-8db31750cb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:34.262594 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4ab21-e5e5-485c-9a80-8db31750cb21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:34.263274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4ab21-e5e5-485c-9a80-8db31750cb21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:34.264937 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4ab21-e5e5-485c-9a80-8db31750cb21 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:34.275892 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1406/2775] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:40:34.281829 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e929a2-6e1d-41d4-9ed2-a61405960695 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:34.293343 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e929a2-6e1d-41d4-9ed2-a61405960695 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:34.298257 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80246c45-0456-45e9-9cd1-d53903ebfe63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:34.298920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80246c45-0456-45e9-9cd1-d53903ebfe63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:34.299198 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80246c45-0456-45e9-9cd1-d53903ebfe63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:34.299607 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80246c45-0456-45e9-9cd1-d53903ebfe63 admin admin] PATH_INFO: `/v3/projects/68b6469f8cb44f119c0a2d3725852581/users/61c03d6f863d4ea1be9b3778f4f5df0a/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:34.316166 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80246c45-0456-45e9-9cd1-d53903ebfe63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68b6469f8cb44f119c0a2d3725852581, user_id=61c03d6f863d4ea1be9b3778f4f5df0a, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:34.318815 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80246c45-0456-45e9-9cd1-d53903ebfe63 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:34.333974 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1370/2776] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:40:34 2022] PUT /identity/v3/projects/68b6469f8cb44f119c0a2d3725852581/users/61c03d6f863d4ea1be9b3778f4f5df0a/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:40:34.344590 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f2379-ada5-4e01-b3da-06d95f5758ce None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:34.345051 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f2379-ada5-4e01-b3da-06d95f5758ce None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:34.345332 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f2379-ada5-4e01-b3da-06d95f5758ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:34.375484 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-316f2379-ada5-4e01-b3da-06d95f5758ce None None] MFA Rules not processed for user `61c03d6f863d4ea1be9b3778f4f5df0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:34.376769 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-316f2379-ada5-4e01-b3da-06d95f5758ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:34.442714 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1407/2777] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:40:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:34.453825 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80246c45-0456-45e9-9cd1-d53903ebfe63 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:34.455550 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-80246c45-0456-45e9-9cd1-d53903ebfe63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:34.477314 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:34.516146 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80246c45-0456-45e9-9cd1-d53903ebfe63 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:34.519762 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad0101-d699-4959-ac00-e03fb0eaae8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:34.520395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad0101-d699-4959-ac00-e03fb0eaae8b service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:34.520675 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad0101-d699-4959-ac00-e03fb0eaae8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:34.520945 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad0101-d699-4959-ac00-e03fb0eaae8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:34.529118 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad0101-d699-4959-ac00-e03fb0eaae8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:34.530544 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad0101-d699-4959-ac00-e03fb0eaae8b service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:34.539120 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1371/2778] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:40:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:40:36.069790 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13518a-2378-4144-a2a7-9ac41a7c20eb None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:36.070315 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13518a-2378-4144-a2a7-9ac41a7c20eb None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:36.070434 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13518a-2378-4144-a2a7-9ac41a7c20eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:36.090358 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-3f13518a-2378-4144-a2a7-9ac41a7c20eb None None] MFA Rules not processed for user `61709ccc1e244185b252179451cd860f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:36.091670 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3f13518a-2378-4144-a2a7-9ac41a7c20eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:36.105907 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:36.152798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1408/2779] 10.222.0.18 () {58 vars in 967 bytes} [Mon Aug 29 20:40:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:36.166421 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad0101-d699-4959-ac00-e03fb0eaae8b service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:36.168044 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-c7ad0101-d699-4959-ac00-e03fb0eaae8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:36.254748 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad0101-d699-4959-ac00-e03fb0eaae8b service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:36.262088 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f5ca3-f051-4a8d-a774-4ac756872092 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599129b7d8ff47efb145787414ddb80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:36.262708 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f5ca3-f051-4a8d-a774-4ac756872092 service swift] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:36.262991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f5ca3-f051-4a8d-a774-4ac756872092 service swift] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:36.263272 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f5ca3-f051-4a8d-a774-4ac756872092 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:36.274175 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f5ca3-f051-4a8d-a774-4ac756872092 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:36.275797 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f5ca3-f051-4a8d-a774-4ac756872092 service swift] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:36.291633 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1372/2780] 10.222.0.18 () {60 vars in 1384 bytes} [Mon Aug 29 20:40:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 29 20:40:44.652696 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13518a-2378-4144-a2a7-9ac41a7c20eb None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:44.717064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13518a-2378-4144-a2a7-9ac41a7c20eb None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:44.721482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a33388-1421-475d-b6f2-24eb04ae6338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:44.722121 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a33388-1421-475d-b6f2-24eb04ae6338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:44.722405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a33388-1421-475d-b6f2-24eb04ae6338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:44.722684 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a33388-1421-475d-b6f2-24eb04ae6338 admin admin] PATH_INFO: `/v3/users/61c03d6f863d4ea1be9b3778f4f5df0a` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:44.724028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a33388-1421-475d-b6f2-24eb04ae6338 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61c03d6f863d4ea1be9b3778f4f5df0a)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:44.725947 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a33388-1421-475d-b6f2-24eb04ae6338 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:44.772788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c5a33388-1421-475d-b6f2-24eb04ae6338 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61c03d6f863d4ea1be9b3778f4f5df0a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:44.780137 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c5a33388-1421-475d-b6f2-24eb04ae6338 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61c03d6f863d4ea1be9b3778f4f5df0a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:44.787924 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c5a33388-1421-475d-b6f2-24eb04ae6338 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61c03d6f863d4ea1be9b3778f4f5df0a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:44.788639 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c5a33388-1421-475d-b6f2-24eb04ae6338 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61c03d6f863d4ea1be9b3778f4f5df0a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:44.789356 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c5a33388-1421-475d-b6f2-24eb04ae6338 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61c03d6f863d4ea1be9b3778f4f5df0a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:44.807888 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1409/2781] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:40:44 2022] DELETE /identity/v3/users/61c03d6f863d4ea1be9b3778f4f5df0a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:40:44.812900 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc6702-4e0c-4918-a60c-9dfb1fa60c71 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:44.813396 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc6702-4e0c-4918-a60c-9dfb1fa60c71 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:44.813496 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc6702-4e0c-4918-a60c-9dfb1fa60c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:44.833318 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-8fcc6702-4e0c-4918-a60c-9dfb1fa60c71 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:44.834524 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8fcc6702-4e0c-4918-a60c-9dfb1fa60c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:44.892833 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1373/2782] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:44.902579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a33388-1421-475d-b6f2-24eb04ae6338 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:44.903987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c5a33388-1421-475d-b6f2-24eb04ae6338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:44.949746 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:44.964614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a33388-1421-475d-b6f2-24eb04ae6338 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:44.968753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fead8-8cbf-49d1-b8b7-979ef2747b66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:44.969549 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fead8-8cbf-49d1-b8b7-979ef2747b66 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:44.969982 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fead8-8cbf-49d1-b8b7-979ef2747b66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:44.970308 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fead8-8cbf-49d1-b8b7-979ef2747b66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:44.981125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34fead8-8cbf-49d1-b8b7-979ef2747b66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:44.982556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34fead8-8cbf-49d1-b8b7-979ef2747b66 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:44.992201 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1410/2783] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:40:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:40:45.276474 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc6702-4e0c-4918-a60c-9dfb1fa60c71 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:45.277973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-8fcc6702-4e0c-4918-a60c-9dfb1fa60c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:45.294308 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcc6702-4e0c-4918-a60c-9dfb1fa60c71 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:45.299995 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a596a-5e4e-4c6d-a765-7a96d152c5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:45.300674 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a596a-5e4e-4c6d-a765-7a96d152c5ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:45.300947 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a596a-5e4e-4c6d-a765-7a96d152c5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:45.301246 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a596a-5e4e-4c6d-a765-7a96d152c5ca admin admin] PATH_INFO: `/v3/projects/68b6469f8cb44f119c0a2d3725852581` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:45.302558 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a596a-5e4e-4c6d-a765-7a96d152c5ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=68b6469f8cb44f119c0a2d3725852581)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:45.304652 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a596a-5e4e-4c6d-a765-7a96d152c5ca admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:45.351413 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:45.353431 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-037a596a-5e4e-4c6d-a765-7a96d152c5ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68b6469f8cb44f119c0a2d3725852581', 'request_id': 'req-037a596a-5e4e-4c6d-a765-7a96d152c5ca'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:45.359841 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-037a596a-5e4e-4c6d-a765-7a96d152c5ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68b6469f8cb44f119c0a2d3725852581', 'request_id': 'req-037a596a-5e4e-4c6d-a765-7a96d152c5ca'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:45.366513 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-037a596a-5e4e-4c6d-a765-7a96d152c5ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68b6469f8cb44f119c0a2d3725852581', 'request_id': 'req-037a596a-5e4e-4c6d-a765-7a96d152c5ca'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:45.377916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-037a596a-5e4e-4c6d-a765-7a96d152c5ca admin admin] The token cache is being invalidate because project 68b6469f8cb44f119c0a2d3725852581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:40:45.378191 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-037a596a-5e4e-4c6d-a765-7a96d152c5ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:45.378835 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-037a596a-5e4e-4c6d-a765-7a96d152c5ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:40:45.380346 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1374/2784] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:40:45 2022] DELETE /identity/v3/projects/68b6469f8cb44f119c0a2d3725852581 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:40:45.385360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c8b92-eab6-4d69-a9dc-c5523ddddbe7 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:45.385648 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c8b92-eab6-4d69-a9dc-c5523ddddbe7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:45.385952 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c8b92-eab6-4d69-a9dc-c5523ddddbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:45.406704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b45c8b92-eab6-4d69-a9dc-c5523ddddbe7 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:45.407835 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b45c8b92-eab6-4d69-a9dc-c5523ddddbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:45.471288 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1411/2785] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:45.481837 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9f482-97c1-4e99-a41d-95f238c04378 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:45.482204 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9f482-97c1-4e99-a41d-95f238c04378 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:45.482511 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9f482-97c1-4e99-a41d-95f238c04378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:45.504806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-fba9f482-97c1-4e99-a41d-95f238c04378 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:45.506014 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-fba9f482-97c1-4e99-a41d-95f238c04378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:45.514245 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1375/2786] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:45.518570 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c8b92-eab6-4d69-a9dc-c5523ddddbe7 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:45.528792 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c8b92-eab6-4d69-a9dc-c5523ddddbe7 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:45.534568 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d359ab2-9adc-4c74-874e-8502600d2d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:45.535180 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d359ab2-9adc-4c74-874e-8502600d2d00 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:45.535463 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d359ab2-9adc-4c74-874e-8502600d2d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:45.535737 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d359ab2-9adc-4c74-874e-8502600d2d00 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:45.536416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d359ab2-9adc-4c74-874e-8502600d2d00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:40:45.536747 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d359ab2-9adc-4c74-874e-8502600d2d00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:45.538336 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d359ab2-9adc-4c74-874e-8502600d2d00 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:45.550578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1412/2787] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:40:45.555129 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9f482-97c1-4e99-a41d-95f238c04378 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:45.564781 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9f482-97c1-4e99-a41d-95f238c04378 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:45.569505 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df3e04-5111-402f-a055-b6447c807526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:45.570089 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df3e04-5111-402f-a055-b6447c807526 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:45.570389 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df3e04-5111-402f-a055-b6447c807526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:45.570660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df3e04-5111-402f-a055-b6447c807526 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:45.571287 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df3e04-5111-402f-a055-b6447c807526 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:45.573196 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df3e04-5111-402f-a055-b6447c807526 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:45.600113 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1376/2788] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:40:45 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:40:45.606431 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d359ab2-9adc-4c74-874e-8502600d2d00 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:45.616437 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d359ab2-9adc-4c74-874e-8502600d2d00 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:45.673371 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc7572-4090-4f8c-b98a-7d93cb603ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:45.674015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc7572-4090-4f8c-b98a-7d93cb603ffc admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:45.674348 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc7572-4090-4f8c-b98a-7d93cb603ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:45.674633 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc7572-4090-4f8c-b98a-7d93cb603ffc admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:45.675278 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc7572-4090-4f8c-b98a-7d93cb603ffc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:45.677222 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc7572-4090-4f8c-b98a-7d93cb603ffc admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:45.699220 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1413/2789] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:40:45 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:40:45.705581 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df3e04-5111-402f-a055-b6447c807526 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:45.716838 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df3e04-5111-402f-a055-b6447c807526 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:45.721454 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df763b40-6a21-4dbf-9c80-9308607e30e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:45.722051 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df763b40-6a21-4dbf-9c80-9308607e30e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:45.722328 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df763b40-6a21-4dbf-9c80-9308607e30e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:45.722600 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df763b40-6a21-4dbf-9c80-9308607e30e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:45.723234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df763b40-6a21-4dbf-9c80-9308607e30e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:45.724851 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df763b40-6a21-4dbf-9c80-9308607e30e5 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:45.736231 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1377/2790] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:40:45.741151 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc7572-4090-4f8c-b98a-7d93cb603ffc admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:45.751042 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc7572-4090-4f8c-b98a-7d93cb603ffc admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:45.755569 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35efc23-6d66-4dbd-a8ea-2a2e1532727a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:45.756199 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35efc23-6d66-4dbd-a8ea-2a2e1532727a admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:45.756482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35efc23-6d66-4dbd-a8ea-2a2e1532727a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:45.756752 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35efc23-6d66-4dbd-a8ea-2a2e1532727a admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:45.757388 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35efc23-6d66-4dbd-a8ea-2a2e1532727a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:45.759465 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35efc23-6d66-4dbd-a8ea-2a2e1532727a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:45.770203 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1414/2791] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:40:45.775017 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df763b40-6a21-4dbf-9c80-9308607e30e5 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:45.785577 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df763b40-6a21-4dbf-9c80-9308607e30e5 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:45.790331 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9bc25-a25c-4239-827c-47168aa7423f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:45.791030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9bc25-a25c-4239-827c-47168aa7423f admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:45.791321 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9bc25-a25c-4239-827c-47168aa7423f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:45.791746 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9bc25-a25c-4239-827c-47168aa7423f admin admin] PATH_INFO: `/v3/projects/732a8a55dfbd44afa826eb6f5571aabf/users/037a2fa230a840cbaf876540632aeafa/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:45.806310 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f9bc25-a25c-4239-827c-47168aa7423f admin admin] RBAC: Authorizing `identity:create_grant(project_id=732a8a55dfbd44afa826eb6f5571aabf, user_id=037a2fa230a840cbaf876540632aeafa, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:45.809042 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f9bc25-a25c-4239-827c-47168aa7423f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:45.834679 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1378/2792] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:40:45 2022] PUT /identity/v3/projects/732a8a55dfbd44afa826eb6f5571aabf/users/037a2fa230a840cbaf876540632aeafa/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:40:45.840556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c56f09-d4f5-40b9-a945-613da77fadc7 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:45.840843 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c56f09-d4f5-40b9-a945-613da77fadc7 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:45.841115 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c56f09-d4f5-40b9-a945-613da77fadc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:45.888766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-a8c56f09-d4f5-40b9-a945-613da77fadc7 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:45.889990 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-a8c56f09-d4f5-40b9-a945-613da77fadc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:45.910748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:45.984560 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1415/2793] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:45.994151 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9bc25-a25c-4239-827c-47168aa7423f admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:45.995952 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d4f9bc25-a25c-4239-827c-47168aa7423f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:46.054299 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9bc25-a25c-4239-827c-47168aa7423f admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:46.057966 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb43f5-d0bf-4eac-b512-b6684db0d38e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:46.058560 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb43f5-d0bf-4eac-b512-b6684db0d38e service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:46.058839 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb43f5-d0bf-4eac-b512-b6684db0d38e service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:46.059130 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb43f5-d0bf-4eac-b512-b6684db0d38e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:46.068274 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb43f5-d0bf-4eac-b512-b6684db0d38e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:46.069691 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb43f5-d0bf-4eac-b512-b6684db0d38e service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:46.079739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1379/2794] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:40:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:40:46.735248 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c25ca-567a-41dc-a958-80124800b24c None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:46.735837 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c25ca-567a-41dc-a958-80124800b24c None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:46.736290 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c25ca-567a-41dc-a958-80124800b24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:46.788318 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-b94c25ca-567a-41dc-a958-80124800b24c None None] MFA Rules not processed for user `037a2fa230a840cbaf876540632aeafa`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:46.790983 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b94c25ca-567a-41dc-a958-80124800b24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:46.881197 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1416/2795] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:40:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:46.893114 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb43f5-d0bf-4eac-b512-b6684db0d38e service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:40:46.894831 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-64bb43f5-d0bf-4eac-b512-b6684db0d38e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:46.969399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:46.983383 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb43f5-d0bf-4eac-b512-b6684db0d38e service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:40:46.987484 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56b094-63c9-4386-b697-63adc4a0450e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:40:46.988149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56b094-63c9-4386-b697-63adc4a0450e service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:46.988432 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56b094-63c9-4386-b697-63adc4a0450e service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:46.988706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56b094-63c9-4386-b697-63adc4a0450e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:46.998291 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56b094-63c9-4386-b697-63adc4a0450e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:40:46.999776 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56b094-63c9-4386-b697-63adc4a0450e service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:40:47.009998 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1380/2796] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:40:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:40:48.378661 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37681611-169e-4158-a43f-14a8a84bad38 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:40:48.379197 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37681611-169e-4158-a43f-14a8a84bad38 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:40:48.379197 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37681611-169e-4158-a43f-14a8a84bad38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:40:48.398165 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-37681611-169e-4158-a43f-14a8a84bad38 None None] MFA Rules not processed for user `037a2fa230a840cbaf876540632aeafa`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:40:48.399849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-37681611-169e-4158-a43f-14a8a84bad38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:40:48.406674 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1417/2797] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:40:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:40:57.005348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=6 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:40:58.392671 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=65 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:41:00.403649 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56b094-63c9-4386-b697-63adc4a0450e service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:00.405094 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6b56b094-63c9-4386-b697-63adc4a0450e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:00.498374 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56b094-63c9-4386-b697-63adc4a0450e service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:00.503644 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852560a-8fa5-44ac-9b0a-03ead5937f88 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0e659eee5e4a9da8d3d7337cb73798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:00.504276 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852560a-8fa5-44ac-9b0a-03ead5937f88 service placement] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:00.504552 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852560a-8fa5-44ac-9b0a-03ead5937f88 service placement] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:00.504819 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852560a-8fa5-44ac-9b0a-03ead5937f88 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:00.506530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5852560a-8fa5-44ac-9b0a-03ead5937f88 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:00.518879 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5852560a-8fa5-44ac-9b0a-03ead5937f88 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:00.520500 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5852560a-8fa5-44ac-9b0a-03ead5937f88 service placement] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:00.520763 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37681611-169e-4158-a43f-14a8a84bad38 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:00.522270 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-37681611-169e-4158-a43f-14a8a84bad38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:00.531363 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1381/2798] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:41:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:41:00.535079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37681611-169e-4158-a43f-14a8a84bad38 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:00.538644 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b370c-efbc-4a41-bb52-b6f17236caad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:00.539250 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b370c-efbc-4a41-bb52-b6f17236caad service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:00.539541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b370c-efbc-4a41-bb52-b6f17236caad service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:00.540455 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b370c-efbc-4a41-bb52-b6f17236caad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:00.542217 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ae1b370c-efbc-4a41-bb52-b6f17236caad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:00.609273 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b370c-efbc-4a41-bb52-b6f17236caad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:00.610700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b370c-efbc-4a41-bb52-b6f17236caad service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:00.620589 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1418/2799] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:41:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:41:06.671549 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852560a-8fa5-44ac-9b0a-03ead5937f88 service placement] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:06.692767 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852560a-8fa5-44ac-9b0a-03ead5937f88 service placement] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:06.701635 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c62159-209e-4d3c-b1a1-fa296e3644bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:06.702255 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c62159-209e-4d3c-b1a1-fa296e3644bd service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:06.702531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c62159-209e-4d3c-b1a1-fa296e3644bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:06.702800 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c62159-209e-4d3c-b1a1-fa296e3644bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:06.704602 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d9c62159-209e-4d3c-b1a1-fa296e3644bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:06.717006 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c62159-209e-4d3c-b1a1-fa296e3644bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:06.718375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c62159-209e-4d3c-b1a1-fa296e3644bd service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:06.729017 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1382/2800] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:41:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:41:10.616590 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=28 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:41:13.480793 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b370c-efbc-4a41-bb52-b6f17236caad service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:13.482382 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ae1b370c-efbc-4a41-bb52-b6f17236caad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:13.499234 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b370c-efbc-4a41-bb52-b6f17236caad service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:13.504021 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cff4a2-4745-4dd1-a9d3-bc683f138f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:13.504707 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cff4a2-4745-4dd1-a9d3-bc683f138f3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:13.504997 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cff4a2-4745-4dd1-a9d3-bc683f138f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:13.505274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cff4a2-4745-4dd1-a9d3-bc683f138f3c admin admin] PATH_INFO: `/v3/users/b93ff419d60043f0aab0c11a6abdaa38` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:13.506416 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cff4a2-4745-4dd1-a9d3-bc683f138f3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b93ff419d60043f0aab0c11a6abdaa38)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:13.509176 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cff4a2-4745-4dd1-a9d3-bc683f138f3c admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:13.559236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e4cff4a2-4745-4dd1-a9d3-bc683f138f3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b93ff419d60043f0aab0c11a6abdaa38'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:13.567098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e4cff4a2-4745-4dd1-a9d3-bc683f138f3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b93ff419d60043f0aab0c11a6abdaa38'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:13.577050 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e4cff4a2-4745-4dd1-a9d3-bc683f138f3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b93ff419d60043f0aab0c11a6abdaa38'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:13.577799 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e4cff4a2-4745-4dd1-a9d3-bc683f138f3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b93ff419d60043f0aab0c11a6abdaa38'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:13.578543 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e4cff4a2-4745-4dd1-a9d3-bc683f138f3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b93ff419d60043f0aab0c11a6abdaa38'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:13.602850 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1419/2801] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:41:13 2022] DELETE /identity/v3/users/b93ff419d60043f0aab0c11a6abdaa38 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:41:13.780436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c62159-209e-4d3c-b1a1-fa296e3644bd service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:13.790807 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c62159-209e-4d3c-b1a1-fa296e3644bd service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:13.852074 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60797c-723c-47da-b11c-f6b9697e9350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:13.852825 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60797c-723c-47da-b11c-f6b9697e9350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:13.853317 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60797c-723c-47da-b11c-f6b9697e9350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:13.853603 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60797c-723c-47da-b11c-f6b9697e9350 admin admin] PATH_INFO: `/v3/projects/2f93e5b4f52b4b8cac7e0336a44b33ba` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:13.854858 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60797c-723c-47da-b11c-f6b9697e9350 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f93e5b4f52b4b8cac7e0336a44b33ba)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:13.856835 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60797c-723c-47da-b11c-f6b9697e9350 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:13.908910 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9a60797c-723c-47da-b11c-f6b9697e9350 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f93e5b4f52b4b8cac7e0336a44b33ba', 'request_id': 'req-9a60797c-723c-47da-b11c-f6b9697e9350'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:13.910163 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:41:13.916275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9a60797c-723c-47da-b11c-f6b9697e9350 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f93e5b4f52b4b8cac7e0336a44b33ba', 'request_id': 'req-9a60797c-723c-47da-b11c-f6b9697e9350'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:13.923857 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9a60797c-723c-47da-b11c-f6b9697e9350 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f93e5b4f52b4b8cac7e0336a44b33ba', 'request_id': 'req-9a60797c-723c-47da-b11c-f6b9697e9350'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:13.935705 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9a60797c-723c-47da-b11c-f6b9697e9350 admin admin] The token cache is being invalidate because project 2f93e5b4f52b4b8cac7e0336a44b33ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:41:13.936157 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9a60797c-723c-47da-b11c-f6b9697e9350 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:13.936808 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-9a60797c-723c-47da-b11c-f6b9697e9350 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:13.938548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1383/2802] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:41:13 2022] DELETE /identity/v3/projects/2f93e5b4f52b4b8cac7e0336a44b33ba => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:41:13.942580 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cff4a2-4745-4dd1-a9d3-bc683f138f3c admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:13.944125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e4cff4a2-4745-4dd1-a9d3-bc683f138f3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:13.957820 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cff4a2-4745-4dd1-a9d3-bc683f138f3c admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:14.016885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e2be2-0fbc-4b50-a934-471dba72c13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:14.017535 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e2be2-0fbc-4b50-a934-471dba72c13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:14.017830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e2be2-0fbc-4b50-a934-471dba72c13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:14.018090 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e2be2-0fbc-4b50-a934-471dba72c13f admin admin] PATH_INFO: `/v3/users/7f0b7046422a44e2acb959ad4878f079` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:14.019258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e2be2-0fbc-4b50-a934-471dba72c13f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f0b7046422a44e2acb959ad4878f079)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:14.021206 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e2be2-0fbc-4b50-a934-471dba72c13f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:14.069604 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-746e2be2-0fbc-4b50-a934-471dba72c13f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f0b7046422a44e2acb959ad4878f079'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:14.077535 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-746e2be2-0fbc-4b50-a934-471dba72c13f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f0b7046422a44e2acb959ad4878f079'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:14.085703 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-746e2be2-0fbc-4b50-a934-471dba72c13f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f0b7046422a44e2acb959ad4878f079'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:14.086418 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-746e2be2-0fbc-4b50-a934-471dba72c13f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f0b7046422a44e2acb959ad4878f079'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:14.087141 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-746e2be2-0fbc-4b50-a934-471dba72c13f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f0b7046422a44e2acb959ad4878f079'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:14.097812 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:41:14.119272 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1420/2803] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:41:13 2022] DELETE /identity/v3/users/7f0b7046422a44e2acb959ad4878f079 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:41:14.292181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60797c-723c-47da-b11c-f6b9697e9350 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:14.293715 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-9a60797c-723c-47da-b11c-f6b9697e9350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:14.308404 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60797c-723c-47da-b11c-f6b9697e9350 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:14.365978 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e15b4-7b4f-433d-b885-26e152ec5db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:14.366637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e15b4-7b4f-433d-b885-26e152ec5db7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:14.366912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e15b4-7b4f-433d-b885-26e152ec5db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:14.367181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e15b4-7b4f-433d-b885-26e152ec5db7 admin admin] PATH_INFO: `/v3/projects/66bffbc642434ef2af36fae3e4a882a4` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:14.368946 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e15b4-7b4f-433d-b885-26e152ec5db7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66bffbc642434ef2af36fae3e4a882a4)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:14.370828 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e15b4-7b4f-433d-b885-26e152ec5db7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:14.415497 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-237e15b4-7b4f-433d-b885-26e152ec5db7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66bffbc642434ef2af36fae3e4a882a4', 'request_id': 'req-237e15b4-7b4f-433d-b885-26e152ec5db7'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:14.420835 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-237e15b4-7b4f-433d-b885-26e152ec5db7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66bffbc642434ef2af36fae3e4a882a4', 'request_id': 'req-237e15b4-7b4f-433d-b885-26e152ec5db7'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:14.428339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-237e15b4-7b4f-433d-b885-26e152ec5db7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66bffbc642434ef2af36fae3e4a882a4', 'request_id': 'req-237e15b4-7b4f-433d-b885-26e152ec5db7'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:14.441224 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-237e15b4-7b4f-433d-b885-26e152ec5db7 admin admin] The token cache is being invalidate because project 66bffbc642434ef2af36fae3e4a882a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:41:14.441507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-237e15b4-7b4f-433d-b885-26e152ec5db7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:14.442146 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-237e15b4-7b4f-433d-b885-26e152ec5db7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:14.443620 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1384/2804] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:41:14 2022] DELETE /identity/v3/projects/66bffbc642434ef2af36fae3e4a882a4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:41:24.119841 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:41:24.438283 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:DELETE=10,keystone:SELECT=57,keystone:INSERT=2 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:41:30.757405 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e2be2-0fbc-4b50-a934-471dba72c13f admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:30.758930 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-746e2be2-0fbc-4b50-a934-471dba72c13f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:30.827571 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e2be2-0fbc-4b50-a934-471dba72c13f admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:30.875436 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ada93-eb68-4857-8645-ecf0014c272b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:30.876145 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ada93-eb68-4857-8645-ecf0014c272b service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:30.876425 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ada93-eb68-4857-8645-ecf0014c272b service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:30.876693 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ada93-eb68-4857-8645-ecf0014c272b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:30.878404 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-957ada93-eb68-4857-8645-ecf0014c272b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:30.941414 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ada93-eb68-4857-8645-ecf0014c272b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:30.942768 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ada93-eb68-4857-8645-ecf0014c272b service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:30.952556 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1421/2805] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:41:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:41:33.499171 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e15b4-7b4f-433d-b885-26e152ec5db7 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:33.502634 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-237e15b4-7b4f-433d-b885-26e152ec5db7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:33.620598 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e15b4-7b4f-433d-b885-26e152ec5db7 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:33.624622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089a55d-655a-44b8-9b52-d82d249df088 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:33.625354 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089a55d-655a-44b8-9b52-d82d249df088 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:33.625712 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089a55d-655a-44b8-9b52-d82d249df088 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:33.626079 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089a55d-655a-44b8-9b52-d82d249df088 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:33.628772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d089a55d-655a-44b8-9b52-d82d249df088 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:33.720841 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089a55d-655a-44b8-9b52-d82d249df088 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:33.723064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089a55d-655a-44b8-9b52-d82d249df088 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:33.740132 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1385/2806] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:41:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:41:34.506610 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ada93-eb68-4857-8645-ecf0014c272b service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:34.508074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-957ada93-eb68-4857-8645-ecf0014c272b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:34.521067 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ada93-eb68-4857-8645-ecf0014c272b service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:34.526750 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dddb3d-c891-4ab1-9c83-9bfe121e579f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:34.527400 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dddb3d-c891-4ab1-9c83-9bfe121e579f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:34.527685 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dddb3d-c891-4ab1-9c83-9bfe121e579f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:34.527987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dddb3d-c891-4ab1-9c83-9bfe121e579f admin admin] PATH_INFO: `/v3/users/068f28eb74e74650a2b80d29664109ae` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:34.529118 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dddb3d-c891-4ab1-9c83-9bfe121e579f admin admin] RBAC: Authorizing `identity:delete_user(user_id=068f28eb74e74650a2b80d29664109ae)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:34.531552 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dddb3d-c891-4ab1-9c83-9bfe121e579f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:34.583549 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-51dddb3d-c891-4ab1-9c83-9bfe121e579f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '068f28eb74e74650a2b80d29664109ae'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:34.585610 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:41:34.592447 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-51dddb3d-c891-4ab1-9c83-9bfe121e579f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '068f28eb74e74650a2b80d29664109ae'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:34.603719 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-51dddb3d-c891-4ab1-9c83-9bfe121e579f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '068f28eb74e74650a2b80d29664109ae'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:34.604391 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-51dddb3d-c891-4ab1-9c83-9bfe121e579f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '068f28eb74e74650a2b80d29664109ae'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:34.605015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-51dddb3d-c891-4ab1-9c83-9bfe121e579f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '068f28eb74e74650a2b80d29664109ae'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:34.622738 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1422/2807] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:41:34 2022] DELETE /identity/v3/users/068f28eb74e74650a2b80d29664109ae => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:41:34.811794 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089a55d-655a-44b8-9b52-d82d249df088 service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:34.826025 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089a55d-655a-44b8-9b52-d82d249df088 service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:34.888169 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab222d8-c9b1-4ee4-845c-c0556d440d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:34.888792 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab222d8-c9b1-4ee4-845c-c0556d440d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:34.889070 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab222d8-c9b1-4ee4-845c-c0556d440d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:34.889340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab222d8-c9b1-4ee4-845c-c0556d440d94 admin admin] PATH_INFO: `/v3/projects/bf00868e84384ee59320184a5e1c46c8` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:34.890452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab222d8-c9b1-4ee4-845c-c0556d440d94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf00868e84384ee59320184a5e1c46c8)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:34.892380 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab222d8-c9b1-4ee4-845c-c0556d440d94 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:34.929530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:41:34.940520 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3ab222d8-c9b1-4ee4-845c-c0556d440d94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf00868e84384ee59320184a5e1c46c8', 'request_id': 'req-3ab222d8-c9b1-4ee4-845c-c0556d440d94'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:34.947561 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3ab222d8-c9b1-4ee4-845c-c0556d440d94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf00868e84384ee59320184a5e1c46c8', 'request_id': 'req-3ab222d8-c9b1-4ee4-845c-c0556d440d94'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:34.956381 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3ab222d8-c9b1-4ee4-845c-c0556d440d94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf00868e84384ee59320184a5e1c46c8', 'request_id': 'req-3ab222d8-c9b1-4ee4-845c-c0556d440d94'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:34.967826 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3ab222d8-c9b1-4ee4-845c-c0556d440d94 admin admin] The token cache is being invalidate because project bf00868e84384ee59320184a5e1c46c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:41:34.968158 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3ab222d8-c9b1-4ee4-845c-c0556d440d94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:34.968813 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-3ab222d8-c9b1-4ee4-845c-c0556d440d94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:41:34.970378 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1386/2808] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:41:34 2022] DELETE /identity/v3/projects/bf00868e84384ee59320184a5e1c46c8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:41:34.975524 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03a07b-dd0f-4145-92fb-d929ea284a63 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:34.975812 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03a07b-dd0f-4145-92fb-d929ea284a63 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:34.976143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03a07b-dd0f-4145-92fb-d929ea284a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:35.025349 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ef03a07b-dd0f-4145-92fb-d929ea284a63 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:41:35.026572 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ef03a07b-dd0f-4145-92fb-d929ea284a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:35.092987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1423/2809] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:41:35.097780 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bc074-cc2d-46b3-88cc-f0fba0344bd8 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:35.098072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bc074-cc2d-46b3-88cc-f0fba0344bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:35.098377 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82bc074-cc2d-46b3-88cc-f0fba0344bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:35.118682 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e82bc074-cc2d-46b3-88cc-f0fba0344bd8 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:41:35.120226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e82bc074-cc2d-46b3-88cc-f0fba0344bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:35.127964 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1387/2810] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:41:35.132214 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03a07b-dd0f-4145-92fb-d929ea284a63 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:35.143274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03a07b-dd0f-4145-92fb-d929ea284a63 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:35.148134 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ca464-0c9c-4173-a8d8-95867dd77d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:35.148748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ca464-0c9c-4173-a8d8-95867dd77d6b admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:35.149029 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ca464-0c9c-4173-a8d8-95867dd77d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:35.149359 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ca464-0c9c-4173-a8d8-95867dd77d6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:35.150044 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ca464-0c9c-4173-a8d8-95867dd77d6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:41:35.150347 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ca464-0c9c-4173-a8d8-95867dd77d6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:35.152112 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ca464-0c9c-4173-a8d8-95867dd77d6b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:35.164058 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1424/2811] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:41:35.168441 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82bc074-cc2d-46b3-88cc-f0fba0344bd8 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:35.177989 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82bc074-cc2d-46b3-88cc-f0fba0344bd8 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:35.183313 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf67988-6861-42dc-b47a-5cd4ff98ea26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:35.183918 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf67988-6861-42dc-b47a-5cd4ff98ea26 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:35.184225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf67988-6861-42dc-b47a-5cd4ff98ea26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:35.184493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf67988-6861-42dc-b47a-5cd4ff98ea26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:35.185117 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf67988-6861-42dc-b47a-5cd4ff98ea26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:35.186995 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf67988-6861-42dc-b47a-5cd4ff98ea26 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:35.219170 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1388/2812] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:41:35 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:41:35.220676 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ca464-0c9c-4173-a8d8-95867dd77d6b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:35.232912 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ca464-0c9c-4173-a8d8-95867dd77d6b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:35.291622 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed43aa6-e111-473a-b68a-f41a2e21042a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:35.292518 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed43aa6-e111-473a-b68a-f41a2e21042a admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:35.292753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed43aa6-e111-473a-b68a-f41a2e21042a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:35.293162 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed43aa6-e111-473a-b68a-f41a2e21042a admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:35.293811 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed43aa6-e111-473a-b68a-f41a2e21042a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:35.295724 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed43aa6-e111-473a-b68a-f41a2e21042a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:35.319131 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1425/2813] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:41:35 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:41:35.323772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf67988-6861-42dc-b47a-5cd4ff98ea26 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:35.340021 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf67988-6861-42dc-b47a-5cd4ff98ea26 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:35.346024 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac515a-927a-4c0e-ad4b-96c0717a121b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:35.346877 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac515a-927a-4c0e-ad4b-96c0717a121b admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:35.347163 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac515a-927a-4c0e-ad4b-96c0717a121b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:35.347438 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac515a-927a-4c0e-ad4b-96c0717a121b admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:35.348118 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac515a-927a-4c0e-ad4b-96c0717a121b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:35.349734 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac515a-927a-4c0e-ad4b-96c0717a121b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:35.360588 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1389/2814] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:41:35.364626 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed43aa6-e111-473a-b68a-f41a2e21042a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:35.375454 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed43aa6-e111-473a-b68a-f41a2e21042a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:35.380108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a88baa-0c77-4b7f-ab5c-5cba43cb6e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:35.380743 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a88baa-0c77-4b7f-ab5c-5cba43cb6e61 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:35.381028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a88baa-0c77-4b7f-ab5c-5cba43cb6e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:35.381305 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a88baa-0c77-4b7f-ab5c-5cba43cb6e61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:35.381954 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a88baa-0c77-4b7f-ab5c-5cba43cb6e61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:35.383537 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a88baa-0c77-4b7f-ab5c-5cba43cb6e61 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:35.394494 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1426/2815] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:41:35.398680 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac515a-927a-4c0e-ad4b-96c0717a121b admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:35.408738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac515a-927a-4c0e-ad4b-96c0717a121b admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:35.413320 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31413a-bcbd-4868-9c21-b61af22d4370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:35.413972 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31413a-bcbd-4868-9c21-b61af22d4370 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:35.414255 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31413a-bcbd-4868-9c21-b61af22d4370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:35.414532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31413a-bcbd-4868-9c21-b61af22d4370 admin admin] PATH_INFO: `/v3/projects/076e4d9f49a944c487077e845726ad82/users/70aaca5187e54f25ad94d9515278e039/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:35.430362 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31413a-bcbd-4868-9c21-b61af22d4370 admin admin] RBAC: Authorizing `identity:create_grant(project_id=076e4d9f49a944c487077e845726ad82, user_id=70aaca5187e54f25ad94d9515278e039, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:35.433660 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c31413a-bcbd-4868-9c21-b61af22d4370 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:35.447573 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1390/2816] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:41:35 2022] PUT /identity/v3/projects/076e4d9f49a944c487077e845726ad82/users/70aaca5187e54f25ad94d9515278e039/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:41:35.452471 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc3d97-a518-4361-94c8-d905765bbeaa None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:35.452758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc3d97-a518-4361-94c8-d905765bbeaa None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:35.453030 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc3d97-a518-4361-94c8-d905765bbeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:35.472190 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-8dcc3d97-a518-4361-94c8-d905765bbeaa None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:41:35.473454 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-8dcc3d97-a518-4361-94c8-d905765bbeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:35.478292 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:41:35.534224 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1427/2817] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:41:35.545233 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31413a-bcbd-4868-9c21-b61af22d4370 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:35.547341 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1c31413a-bcbd-4868-9c21-b61af22d4370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:35.607415 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31413a-bcbd-4868-9c21-b61af22d4370 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:35.611216 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59ac4e-1f96-43fd-8838-5d3ed121d8e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:35.619397 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59ac4e-1f96-43fd-8838-5d3ed121d8e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:35.619397 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59ac4e-1f96-43fd-8838-5d3ed121d8e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:35.619397 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59ac4e-1f96-43fd-8838-5d3ed121d8e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:35.622464 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b59ac4e-1f96-43fd-8838-5d3ed121d8e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:35.623925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b59ac4e-1f96-43fd-8838-5d3ed121d8e4 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:35.633668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1391/2818] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:41:39.949900 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb568d-638b-4588-941c-a9f504ba85ff None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:39.951094 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb568d-638b-4588-941c-a9f504ba85ff None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:39.951541 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb568d-638b-4588-941c-a9f504ba85ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:39.988108 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-9cbb568d-638b-4588-941c-a9f504ba85ff None None] MFA Rules not processed for user `70aaca5187e54f25ad94d9515278e039`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:41:39.989348 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9cbb568d-638b-4588-941c-a9f504ba85ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:40.066185 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1428/2819] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:41:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:41:40.081742 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59ac4e-1f96-43fd-8838-5d3ed121d8e4 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:41:40.083370 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1b59ac4e-1f96-43fd-8838-5d3ed121d8e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:40.163243 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:41:40.210343 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59ac4e-1f96-43fd-8838-5d3ed121d8e4 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:41:40.219712 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34875dfc-e541-4d07-809d-5fcebafe18d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:41:40.220435 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34875dfc-e541-4d07-809d-5fcebafe18d8 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:40.220793 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34875dfc-e541-4d07-809d-5fcebafe18d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:40.221071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34875dfc-e541-4d07-809d-5fcebafe18d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:40.231888 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34875dfc-e541-4d07-809d-5fcebafe18d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:41:40.233569 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34875dfc-e541-4d07-809d-5fcebafe18d8 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:41:40.244251 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1392/2820] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:41:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:41:41.758003 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7066c-d9cb-403a-bfd6-61351635f692 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:41.758523 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7066c-d9cb-403a-bfd6-61351635f692 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:41.758831 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7066c-d9cb-403a-bfd6-61351635f692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:41.780704 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d4f7066c-d9cb-403a-bfd6-61351635f692 None None] MFA Rules not processed for user `70aaca5187e54f25ad94d9515278e039`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:41:41.781939 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d4f7066c-d9cb-403a-bfd6-61351635f692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:41.790601 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1429/2821] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:41:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:41:43.023369 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b8c03-5cd8-4936-9608-e87c672a6c38 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:41:43.023968 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b8c03-5cd8-4936-9608-e87c672a6c38 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:41:43.024073 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b8c03-5cd8-4936-9608-e87c672a6c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:41:43.046072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-222b8c03-5cd8-4936-9608-e87c672a6c38 None None] MFA Rules not processed for user `fca06f41184444ccb0ddb57e5d44f227`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:41:43.047892 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-222b8c03-5cd8-4936-9608-e87c672a6c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:41:43.120788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1393/2822] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:41:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:41:51.775638 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=58 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:41:53.111651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=42 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:42:01.706614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e80b2-875b-427a-8ecc-55a1e5b88f98 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:42:01.710363 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e80b2-875b-427a-8ecc-55a1e5b88f98 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:42:01.714681 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e80b2-875b-427a-8ecc-55a1e5b88f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:42:01.742577 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-9a7e80b2-875b-427a-8ecc-55a1e5b88f98 None None] MFA Rules not processed for user `037a2fa230a840cbaf876540632aeafa`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:42:01.743903 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-9a7e80b2-875b-427a-8ecc-55a1e5b88f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:42:01.818074 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1430/2823] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:42:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:42:05.576334 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b8c03-5cd8-4936-9608-e87c672a6c38 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:42:05.591849 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b8c03-5cd8-4936-9608-e87c672a6c38 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:42:05.597322 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a560b-c01a-49a0-bb99-749483b8ed18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:42:05.598265 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a560b-c01a-49a0-bb99-749483b8ed18 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:42:05.598705 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a560b-c01a-49a0-bb99-749483b8ed18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:42:05.599154 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a560b-c01a-49a0-bb99-749483b8ed18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:42:05.601863 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-645a560b-c01a-49a0-bb99-749483b8ed18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:42:05.685347 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a560b-c01a-49a0-bb99-749483b8ed18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:42:05.687149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a560b-c01a-49a0-bb99-749483b8ed18 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:42:05.698532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1394/2824] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:42:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:42:11.806102 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=25 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:42:15.693646 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=30 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:43:00.440946 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c49641-0b4a-4756-877f-b394dc51ef20 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:43:00.441551 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c49641-0b4a-4756-877f-b394dc51ef20 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:43:00.441551 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c49641-0b4a-4756-877f-b394dc51ef20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:43:00.460103 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-e7c49641-0b4a-4756-877f-b394dc51ef20 None None] MFA Rules not processed for user `70aaca5187e54f25ad94d9515278e039`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:43:00.461304 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e7c49641-0b4a-4756-877f-b394dc51ef20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:43:00.468588 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1431/2825] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:43:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:43:00.509071 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a560b-c01a-49a0-bb99-749483b8ed18 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:43:00.510469 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-645a560b-c01a-49a0-bb99-749483b8ed18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:43:00.567153 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a560b-c01a-49a0-bb99-749483b8ed18 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:43:00.570753 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8f293-e99e-4336-ae9d-e200ea210f6c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d022ed8d2ac43ae816aa396a8e2f17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:43:00.571345 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8f293-e99e-4336-ae9d-e200ea210f6c service glance] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:43:00.571626 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8f293-e99e-4336-ae9d-e200ea210f6c service glance] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:43:00.572578 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8f293-e99e-4336-ae9d-e200ea210f6c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:43:00.574453 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-14c8f293-e99e-4336-ae9d-e200ea210f6c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:43:00.588616 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8f293-e99e-4336-ae9d-e200ea210f6c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:43:00.590064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8f293-e99e-4336-ae9d-e200ea210f6c service glance] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:43:00.602098 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1395/2826] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:43:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:43:01.211835 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c49641-0b4a-4756-877f-b394dc51ef20 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:43:01.221638 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c49641-0b4a-4756-877f-b394dc51ef20 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:43:01.225080 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c63bf-b5a8-4301-be6c-f0505bf0136c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:43:01.225677 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c63bf-b5a8-4301-be6c-f0505bf0136c service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:43:01.225956 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c63bf-b5a8-4301-be6c-f0505bf0136c service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:43:01.226263 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c63bf-b5a8-4301-be6c-f0505bf0136c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:43:01.228006 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-850c63bf-b5a8-4301-be6c-f0505bf0136c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:43:01.241542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c63bf-b5a8-4301-be6c-f0505bf0136c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:43:01.242958 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c63bf-b5a8-4301-be6c-f0505bf0136c service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:43:01.254946 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1432/2827] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:43:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:43:02.712211 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8f293-e99e-4336-ae9d-e200ea210f6c service glance] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:43:02.713736 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-14c8f293-e99e-4336-ae9d-e200ea210f6c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:43:02.726921 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8f293-e99e-4336-ae9d-e200ea210f6c service glance] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:43:02.731853 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9a63b-15b7-4a41-93af-313463f3772a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:43:02.732750 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9a63b-15b7-4a41-93af-313463f3772a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:43:02.732916 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9a63b-15b7-4a41-93af-313463f3772a admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:43:02.733383 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9a63b-15b7-4a41-93af-313463f3772a admin admin] PATH_INFO: `/v3/users/037a2fa230a840cbaf876540632aeafa` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:43:02.734534 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9a63b-15b7-4a41-93af-313463f3772a admin admin] RBAC: Authorizing `identity:delete_user(user_id=037a2fa230a840cbaf876540632aeafa)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:43:02.736492 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9a63b-15b7-4a41-93af-313463f3772a admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:43:02.793072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-86a9a63b-15b7-4a41-93af-313463f3772a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '037a2fa230a840cbaf876540632aeafa'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:43:02.800301 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-86a9a63b-15b7-4a41-93af-313463f3772a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '037a2fa230a840cbaf876540632aeafa'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:43:02.807520 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-86a9a63b-15b7-4a41-93af-313463f3772a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '037a2fa230a840cbaf876540632aeafa'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:43:02.808237 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-86a9a63b-15b7-4a41-93af-313463f3772a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '037a2fa230a840cbaf876540632aeafa'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:43:02.821076 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-86a9a63b-15b7-4a41-93af-313463f3772a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '037a2fa230a840cbaf876540632aeafa'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:43:02.827829 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1396/2828] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:43:02 2022] DELETE /identity/v3/users/037a2fa230a840cbaf876540632aeafa => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:43:03.149807 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c63bf-b5a8-4301-be6c-f0505bf0136c service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:43:03.171252 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c63bf-b5a8-4301-be6c-f0505bf0136c service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:43:03.241555 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a20d21-b664-4954-94b5-afd599d44385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:43:03.242204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a20d21-b664-4954-94b5-afd599d44385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:43:03.242487 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a20d21-b664-4954-94b5-afd599d44385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:43:03.242760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a20d21-b664-4954-94b5-afd599d44385 admin admin] PATH_INFO: `/v3/projects/732a8a55dfbd44afa826eb6f5571aabf` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:43:03.243879 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a20d21-b664-4954-94b5-afd599d44385 admin admin] RBAC: Authorizing `identity:delete_project(project_id=732a8a55dfbd44afa826eb6f5571aabf)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:43:03.245917 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a20d21-b664-4954-94b5-afd599d44385 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:43:03.294529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b6a20d21-b664-4954-94b5-afd599d44385 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '732a8a55dfbd44afa826eb6f5571aabf', 'request_id': 'req-b6a20d21-b664-4954-94b5-afd599d44385'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:43:03.307212 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b6a20d21-b664-4954-94b5-afd599d44385 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '732a8a55dfbd44afa826eb6f5571aabf', 'request_id': 'req-b6a20d21-b664-4954-94b5-afd599d44385'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:43:03.313661 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b6a20d21-b664-4954-94b5-afd599d44385 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '732a8a55dfbd44afa826eb6f5571aabf', 'request_id': 'req-b6a20d21-b664-4954-94b5-afd599d44385'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:43:03.325764 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b6a20d21-b664-4954-94b5-afd599d44385 admin admin] The token cache is being invalidate because project 732a8a55dfbd44afa826eb6f5571aabf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:43:03.326218 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b6a20d21-b664-4954-94b5-afd599d44385 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:43:03.326997 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-b6a20d21-b664-4954-94b5-afd599d44385 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:43:03.328615 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1433/2829] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:43:03 2022] DELETE /identity/v3/projects/732a8a55dfbd44afa826eb6f5571aabf => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:43:12.823533 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:43:13.321984 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:44:00.252424 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9a63b-15b7-4a41-93af-313463f3772a admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:44:00.254095 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-86a9a63b-15b7-4a41-93af-313463f3772a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:44:00.369536 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9a63b-15b7-4a41-93af-313463f3772a admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:44:00.483391 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd979108-f979-43fb-8299-5118c85ce876 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:44:00.484455 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd979108-f979-43fb-8299-5118c85ce876 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:44:00.484975 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd979108-f979-43fb-8299-5118c85ce876 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:44:00.486932 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd979108-f979-43fb-8299-5118c85ce876 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:44:00.490019 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-cd979108-f979-43fb-8299-5118c85ce876 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:44:00.503966 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd979108-f979-43fb-8299-5118c85ce876 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:44:00.505757 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd979108-f979-43fb-8299-5118c85ce876 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:44:00.521048 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1397/2830] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:44:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:44:01.177812 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a20d21-b664-4954-94b5-afd599d44385 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:44:01.180268 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-b6a20d21-b664-4954-94b5-afd599d44385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:44:01.200807 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a20d21-b664-4954-94b5-afd599d44385 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:44:01.208737 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827bbed-668a-499f-974f-624d287a13b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:44:01.209852 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827bbed-668a-499f-974f-624d287a13b8 service nova] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:44:01.210700 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827bbed-668a-499f-974f-624d287a13b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:44:01.211160 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827bbed-668a-499f-974f-624d287a13b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:44:01.218211 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5827bbed-668a-499f-974f-624d287a13b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:44:01.238897 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827bbed-668a-499f-974f-624d287a13b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:44:01.241249 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827bbed-668a-499f-974f-624d287a13b8 service nova] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:44:01.271914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1434/2831] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:44:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:44:01.350263 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd979108-f979-43fb-8299-5118c85ce876 service nova] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:44:01.351991 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-cd979108-f979-43fb-8299-5118c85ce876 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:44:01.426065 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd979108-f979-43fb-8299-5118c85ce876 service nova] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:44:01.430563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7b661-0098-44b5-ad2b-bc538f4041d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:44:01.432019 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7b661-0098-44b5-ad2b-bc538f4041d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:44:01.432906 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7b661-0098-44b5-ad2b-bc538f4041d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:44:01.433491 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7b661-0098-44b5-ad2b-bc538f4041d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:44:01.436786 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f2a7b661-0098-44b5-ad2b-bc538f4041d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:44:01.455855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a7b661-0098-44b5-ad2b-bc538f4041d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:44:01.457322 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a7b661-0098-44b5-ad2b-bc538f4041d5 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:44:01.468840 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1398/2832] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:44:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:44:11.249893 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:44:11.463686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=88 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:44:12.933373 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410abb9-5fcb-4248-a3c2-4be929d65e72 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:44:12.933965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410abb9-5fcb-4248-a3c2-4be929d65e72 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:44:12.933965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410abb9-5fcb-4248-a3c2-4be929d65e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:44:12.954184 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-f410abb9-5fcb-4248-a3c2-4be929d65e72 None None] MFA Rules not processed for user `70aaca5187e54f25ad94d9515278e039`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:44:12.955395 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f410abb9-5fcb-4248-a3c2-4be929d65e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:44:13.009803 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1435/2833] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:44:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:44:13.319254 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7b661-0098-44b5-ad2b-bc538f4041d5 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:44:13.320678 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-f2a7b661-0098-44b5-ad2b-bc538f4041d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:44:13.330896 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7b661-0098-44b5-ad2b-bc538f4041d5 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:44:13.334486 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0ce97-124f-4b4f-abf8-e88dafe4942c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:44:13.335119 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0ce97-124f-4b4f-abf8-e88dafe4942c service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:44:13.335395 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0ce97-124f-4b4f-abf8-e88dafe4942c service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:44:13.335665 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0ce97-124f-4b4f-abf8-e88dafe4942c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:44:13.337437 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-16e0ce97-124f-4b4f-abf8-e88dafe4942c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:44:13.346863 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0ce97-124f-4b4f-abf8-e88dafe4942c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:44:13.348302 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0ce97-124f-4b4f-abf8-e88dafe4942c service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:44:13.357053 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1399/2834] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:44:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:44:15.465628 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410abb9-5fcb-4248-a3c2-4be929d65e72 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:44:15.475441 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410abb9-5fcb-4248-a3c2-4be929d65e72 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:44:15.479003 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e8910-6bae-42af-8e14-2ed114bfb5f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:44:15.479607 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e8910-6bae-42af-8e14-2ed114bfb5f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:44:15.479913 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e8910-6bae-42af-8e14-2ed114bfb5f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:44:15.480219 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e8910-6bae-42af-8e14-2ed114bfb5f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:44:15.481924 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6c4e8910-6bae-42af-8e14-2ed114bfb5f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:44:15.545189 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e8910-6bae-42af-8e14-2ed114bfb5f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:44:15.546741 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e8910-6bae-42af-8e14-2ed114bfb5f1 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:44:15.556410 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1436/2835] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:44:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:44:15.702803 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0ce97-124f-4b4f-abf8-e88dafe4942c service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:44:15.704347 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-16e0ce97-124f-4b4f-abf8-e88dafe4942c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:44:15.790693 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0ce97-124f-4b4f-abf8-e88dafe4942c service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:44:15.795954 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7be2ae-8b79-439c-8bb7-1984e8b5d617 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0e659eee5e4a9da8d3d7337cb73798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:44:15.796579 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7be2ae-8b79-439c-8bb7-1984e8b5d617 service placement] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:44:15.798021 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7be2ae-8b79-439c-8bb7-1984e8b5d617 service placement] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:44:15.798021 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7be2ae-8b79-439c-8bb7-1984e8b5d617 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:44:15.798936 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4f7be2ae-8b79-439c-8bb7-1984e8b5d617 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:44:15.812988 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7be2ae-8b79-439c-8bb7-1984e8b5d617 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:44:15.815428 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7be2ae-8b79-439c-8bb7-1984e8b5d617 service placement] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:44:15.827387 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1400/2836] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:44:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:44:18.729350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e8910-6bae-42af-8e14-2ed114bfb5f1 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:44:18.730902 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-6c4e8910-6bae-42af-8e14-2ed114bfb5f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:44:18.746221 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e8910-6bae-42af-8e14-2ed114bfb5f1 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:44:18.752396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ce3d3-ccfa-4db3-b4cb-b6bae199813e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0e659eee5e4a9da8d3d7337cb73798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:44:18.753047 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ce3d3-ccfa-4db3-b4cb-b6bae199813e service placement] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:44:18.753331 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ce3d3-ccfa-4db3-b4cb-b6bae199813e service placement] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:44:18.753608 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ce3d3-ccfa-4db3-b4cb-b6bae199813e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:44:18.755422 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-5d6ce3d3-ccfa-4db3-b4cb-b6bae199813e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:44:18.769210 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ce3d3-ccfa-4db3-b4cb-b6bae199813e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:44:18.771734 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ce3d3-ccfa-4db3-b4cb-b6bae199813e service placement] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:44:18.789995 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1437/2837] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:44:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:44:22.509624 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7be2ae-8b79-439c-8bb7-1984e8b5d617 service placement] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:44:22.511011 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-4f7be2ae-8b79-439c-8bb7-1984e8b5d617 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:44:22.526399 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7be2ae-8b79-439c-8bb7-1984e8b5d617 service placement] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:44:22.530847 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d8bf7-0757-4bf0-abb5-108fb4995a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:44:22.531467 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d8bf7-0757-4bf0-abb5-108fb4995a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:44:22.531742 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d8bf7-0757-4bf0-abb5-108fb4995a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:44:22.532039 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d8bf7-0757-4bf0-abb5-108fb4995a22 admin admin] PATH_INFO: `/v3/users/fca06f41184444ccb0ddb57e5d44f227` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:44:22.533132 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d8bf7-0757-4bf0-abb5-108fb4995a22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fca06f41184444ccb0ddb57e5d44f227)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:44:22.535018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d8bf7-0757-4bf0-abb5-108fb4995a22 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:44:22.583284 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-437d8bf7-0757-4bf0-abb5-108fb4995a22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fca06f41184444ccb0ddb57e5d44f227'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:44:22.590870 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-437d8bf7-0757-4bf0-abb5-108fb4995a22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fca06f41184444ccb0ddb57e5d44f227'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:44:22.598451 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-437d8bf7-0757-4bf0-abb5-108fb4995a22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fca06f41184444ccb0ddb57e5d44f227'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:44:22.599070 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-437d8bf7-0757-4bf0-abb5-108fb4995a22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fca06f41184444ccb0ddb57e5d44f227'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:44:22.611867 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-437d8bf7-0757-4bf0-abb5-108fb4995a22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fca06f41184444ccb0ddb57e5d44f227'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:44:22.618488 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1401/2838] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:44:22 2022] DELETE /identity/v3/users/fca06f41184444ccb0ddb57e5d44f227 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:44:22.802650 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ce3d3-ccfa-4db3-b4cb-b6bae199813e service placement] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:44:22.816287 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ce3d3-ccfa-4db3-b4cb-b6bae199813e service placement] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:44:22.881204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b551f-0237-4b83-9e33-1fec0f2bd831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:44:22.882533 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b551f-0237-4b83-9e33-1fec0f2bd831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:44:22.882844 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b551f-0237-4b83-9e33-1fec0f2bd831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:44:22.883139 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b551f-0237-4b83-9e33-1fec0f2bd831 admin admin] PATH_INFO: `/v3/projects/4ac573b4aa7a47c49d129afd9939e26d` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:44:22.884411 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b551f-0237-4b83-9e33-1fec0f2bd831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ac573b4aa7a47c49d129afd9939e26d)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:44:22.886546 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b551f-0237-4b83-9e33-1fec0f2bd831 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:44:22.925925 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:44:22.944251 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-153b551f-0237-4b83-9e33-1fec0f2bd831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ac573b4aa7a47c49d129afd9939e26d', 'request_id': 'req-153b551f-0237-4b83-9e33-1fec0f2bd831'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:44:22.952655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-153b551f-0237-4b83-9e33-1fec0f2bd831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ac573b4aa7a47c49d129afd9939e26d', 'request_id': 'req-153b551f-0237-4b83-9e33-1fec0f2bd831'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:44:22.958985 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-153b551f-0237-4b83-9e33-1fec0f2bd831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ac573b4aa7a47c49d129afd9939e26d', 'request_id': 'req-153b551f-0237-4b83-9e33-1fec0f2bd831'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:44:22.970123 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-153b551f-0237-4b83-9e33-1fec0f2bd831 admin admin] The token cache is being invalidate because project 4ac573b4aa7a47c49d129afd9939e26d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:44:22.970431 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-153b551f-0237-4b83-9e33-1fec0f2bd831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:44:22.971109 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-153b551f-0237-4b83-9e33-1fec0f2bd831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:44:22.972646 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1438/2839] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:44:22 2022] DELETE /identity/v3/projects/4ac573b4aa7a47c49d129afd9939e26d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:44:22.977184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d8bf7-0757-4bf0-abb5-108fb4995a22 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:44:22.978587 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-437d8bf7-0757-4bf0-abb5-108fb4995a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:44:22.992064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d8bf7-0757-4bf0-abb5-108fb4995a22 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:44:23.036261 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:44:23.051659 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa87a76-db50-4d55-ac20-c106a6d82022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:44:23.052348 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa87a76-db50-4d55-ac20-c106a6d82022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:44:23.052630 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa87a76-db50-4d55-ac20-c106a6d82022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:44:23.052925 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa87a76-db50-4d55-ac20-c106a6d82022 admin admin] PATH_INFO: `/v3/users/9a07538a219c4182b0a21d157a90c60d` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:44:23.054680 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa87a76-db50-4d55-ac20-c106a6d82022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a07538a219c4182b0a21d157a90c60d)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:44:23.056702 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa87a76-db50-4d55-ac20-c106a6d82022 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:44:23.110046 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-caa87a76-db50-4d55-ac20-c106a6d82022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a07538a219c4182b0a21d157a90c60d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:44:23.118672 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-caa87a76-db50-4d55-ac20-c106a6d82022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a07538a219c4182b0a21d157a90c60d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:44:23.126012 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-caa87a76-db50-4d55-ac20-c106a6d82022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a07538a219c4182b0a21d157a90c60d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:44:23.126632 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-caa87a76-db50-4d55-ac20-c106a6d82022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a07538a219c4182b0a21d157a90c60d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:44:23.138481 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-caa87a76-db50-4d55-ac20-c106a6d82022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a07538a219c4182b0a21d157a90c60d'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:44:23.144375 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1402/2840] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:44:22 2022] DELETE /identity/v3/users/9a07538a219c4182b0a21d157a90c60d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:44:23.373501 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b551f-0237-4b83-9e33-1fec0f2bd831 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:44:23.375472 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-153b551f-0237-4b83-9e33-1fec0f2bd831 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:44:23.399164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b551f-0237-4b83-9e33-1fec0f2bd831 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:44:23.472125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd790d-00e8-40e6-813d-719d77814775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:44:23.472748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd790d-00e8-40e6-813d-719d77814775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:44:23.473034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd790d-00e8-40e6-813d-719d77814775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:44:23.473311 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd790d-00e8-40e6-813d-719d77814775 admin admin] PATH_INFO: `/v3/projects/f885e578ed46463dad421c9f1ecc6520` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:44:23.474460 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd790d-00e8-40e6-813d-719d77814775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f885e578ed46463dad421c9f1ecc6520)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:44:23.477144 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd790d-00e8-40e6-813d-719d77814775 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:44:23.527126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-62cd790d-00e8-40e6-813d-719d77814775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f885e578ed46463dad421c9f1ecc6520', 'request_id': 'req-62cd790d-00e8-40e6-813d-719d77814775'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:44:23.535858 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-62cd790d-00e8-40e6-813d-719d77814775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f885e578ed46463dad421c9f1ecc6520', 'request_id': 'req-62cd790d-00e8-40e6-813d-719d77814775'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:44:23.542360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-62cd790d-00e8-40e6-813d-719d77814775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f885e578ed46463dad421c9f1ecc6520', 'request_id': 'req-62cd790d-00e8-40e6-813d-719d77814775'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:44:23.554578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-62cd790d-00e8-40e6-813d-719d77814775 admin admin] The token cache is being invalidate because project f885e578ed46463dad421c9f1ecc6520 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81398) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:44:23.554882 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-62cd790d-00e8-40e6-813d-719d77814775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:44:23.555543 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-62cd790d-00e8-40e6-813d-719d77814775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:44:23.557345 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1439/2841] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:44:23 2022] DELETE /identity/v3/projects/f885e578ed46463dad421c9f1ecc6520 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:44:25.912176 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa87a76-db50-4d55-ac20-c106a6d82022 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:44:25.914788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-caa87a76-db50-4d55-ac20-c106a6d82022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:44:25.997679 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa87a76-db50-4d55-ac20-c106a6d82022 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:44:26.051301 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:44:26.060220 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2903f-95e0-4536-a695-8b56c71e2686 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c0c44754a1c04a2b87081aba6cb509d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:44:26.061011 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2903f-95e0-4536-a695-8b56c71e2686 service nova] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:44:26.061300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2903f-95e0-4536-a695-8b56c71e2686 service nova] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:44:26.061761 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2903f-95e0-4536-a695-8b56c71e2686 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:44:26.063815 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-12a2903f-95e0-4536-a695-8b56c71e2686 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:44:26.076650 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2903f-95e0-4536-a695-8b56c71e2686 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:44:26.078283 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2903f-95e0-4536-a695-8b56c71e2686 service nova] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:44:26.088656 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1403/2842] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:44:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:44:33.550844 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=65,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:44:36.084266 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=4 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:45:21.764622 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e29322-8442-449c-9e77-bd572c12da8d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:45:21.765198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e29322-8442-449c-9e77-bd572c12da8d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:45:21.765198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e29322-8442-449c-9e77-bd572c12da8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:45:21.784818 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-52e29322-8442-449c-9e77-bd572c12da8d None None] MFA Rules not processed for user `70aaca5187e54f25ad94d9515278e039`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:45:21.786028 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-52e29322-8442-449c-9e77-bd572c12da8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:45:21.844780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1440/2843] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:45:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:45:22.185470 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2903f-95e0-4536-a695-8b56c71e2686 service nova] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:45:22.186860 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-12a2903f-95e0-4536-a695-8b56c71e2686 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:45:22.242663 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2903f-95e0-4536-a695-8b56c71e2686 service nova] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:45:22.246075 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098a070-dd1c-4c87-a920-fb347a3e35aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:45:22.246668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098a070-dd1c-4c87-a920-fb347a3e35aa service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:45:22.246943 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098a070-dd1c-4c87-a920-fb347a3e35aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:45:22.247215 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098a070-dd1c-4c87-a920-fb347a3e35aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:45:22.248939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-2098a070-dd1c-4c87-a920-fb347a3e35aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:45:22.258612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098a070-dd1c-4c87-a920-fb347a3e35aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:45:22.260135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2098a070-dd1c-4c87-a920-fb347a3e35aa service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:45:22.268378 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1404/2844] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:45:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:45:31.838145 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=25 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:45:32.264752 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=26 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:46:04.922350 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e29322-8442-449c-9e77-bd572c12da8d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:04.923753 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-52e29322-8442-449c-9e77-bd572c12da8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:05.004164 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e29322-8442-449c-9e77-bd572c12da8d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:05.009625 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e6aa6-6d0c-4066-9472-8995ad8a8983 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0e659eee5e4a9da8d3d7337cb73798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:05.010260 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e6aa6-6d0c-4066-9472-8995ad8a8983 service placement] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:05.010542 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e6aa6-6d0c-4066-9472-8995ad8a8983 service placement] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:05.010826 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e6aa6-6d0c-4066-9472-8995ad8a8983 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:05.012656 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-bd3e6aa6-6d0c-4066-9472-8995ad8a8983 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:05.026360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e6aa6-6d0c-4066-9472-8995ad8a8983 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:05.027966 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e6aa6-6d0c-4066-9472-8995ad8a8983 service placement] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:05.038758 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1441/2845] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:46:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:46:15.033819 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=32 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:46:29.872817 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098a070-dd1c-4c87-a920-fb347a3e35aa service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:29.885389 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2098a070-dd1c-4c87-a920-fb347a3e35aa service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:29.890710 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8fe5a-9045-43a0-a80d-c71728ae7ba7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0e659eee5e4a9da8d3d7337cb73798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:29.891316 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8fe5a-9045-43a0-a80d-c71728ae7ba7 service placement] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:29.891596 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8fe5a-9045-43a0-a80d-c71728ae7ba7 service placement] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:29.891874 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8fe5a-9045-43a0-a80d-c71728ae7ba7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:29.893850 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7ed8fe5a-9045-43a0-a80d-c71728ae7ba7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:29.906147 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8fe5a-9045-43a0-a80d-c71728ae7ba7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:29.907713 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8fe5a-9045-43a0-a80d-c71728ae7ba7 service placement] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:29.919386 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1405/2846] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:46:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:46:29.937160 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e6aa6-6d0c-4066-9472-8995ad8a8983 service placement] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:29.938754 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-bd3e6aa6-6d0c-4066-9472-8995ad8a8983 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:30.012795 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e6aa6-6d0c-4066-9472-8995ad8a8983 service placement] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:30.017293 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3405a-152c-4a1f-8dac-5cea0a835cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:30.017924 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3405a-152c-4a1f-8dac-5cea0a835cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:30.018204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3405a-152c-4a1f-8dac-5cea0a835cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:30.018478 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3405a-152c-4a1f-8dac-5cea0a835cb3 admin admin] PATH_INFO: `/v3/users/70aaca5187e54f25ad94d9515278e039` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:30.019578 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3405a-152c-4a1f-8dac-5cea0a835cb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70aaca5187e54f25ad94d9515278e039)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:30.022034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3405a-152c-4a1f-8dac-5cea0a835cb3 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:30.087079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-70e3405a-152c-4a1f-8dac-5cea0a835cb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70aaca5187e54f25ad94d9515278e039'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:30.097458 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-70e3405a-152c-4a1f-8dac-5cea0a835cb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70aaca5187e54f25ad94d9515278e039'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:30.106239 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-70e3405a-152c-4a1f-8dac-5cea0a835cb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70aaca5187e54f25ad94d9515278e039'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:30.106933 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-70e3405a-152c-4a1f-8dac-5cea0a835cb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70aaca5187e54f25ad94d9515278e039'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:30.107551 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-70e3405a-152c-4a1f-8dac-5cea0a835cb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70aaca5187e54f25ad94d9515278e039'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:30.129076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1442/2847] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:46:29 2022] DELETE /identity/v3/users/70aaca5187e54f25ad94d9515278e039 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:46:30.323772 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8fe5a-9045-43a0-a80d-c71728ae7ba7 service placement] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:30.336416 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8fe5a-9045-43a0-a80d-c71728ae7ba7 service placement] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:30.403915 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6340910-eb21-466c-8b35-fe6f7d5149b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:30.404956 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6340910-eb21-466c-8b35-fe6f7d5149b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:30.405235 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6340910-eb21-466c-8b35-fe6f7d5149b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:30.405507 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6340910-eb21-466c-8b35-fe6f7d5149b0 admin admin] PATH_INFO: `/v3/projects/076e4d9f49a944c487077e845726ad82` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:30.406648 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6340910-eb21-466c-8b35-fe6f7d5149b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=076e4d9f49a944c487077e845726ad82)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:30.408588 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6340910-eb21-466c-8b35-fe6f7d5149b0 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:30.462612 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d6340910-eb21-466c-8b35-fe6f7d5149b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '076e4d9f49a944c487077e845726ad82', 'request_id': 'req-d6340910-eb21-466c-8b35-fe6f7d5149b0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:30.469002 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d6340910-eb21-466c-8b35-fe6f7d5149b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '076e4d9f49a944c487077e845726ad82', 'request_id': 'req-d6340910-eb21-466c-8b35-fe6f7d5149b0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:30.477846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d6340910-eb21-466c-8b35-fe6f7d5149b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '076e4d9f49a944c487077e845726ad82', 'request_id': 'req-d6340910-eb21-466c-8b35-fe6f7d5149b0'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:30.490018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d6340910-eb21-466c-8b35-fe6f7d5149b0 admin admin] The token cache is being invalidate because project 076e4d9f49a944c487077e845726ad82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:46:30.490289 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d6340910-eb21-466c-8b35-fe6f7d5149b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:30.490963 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-d6340910-eb21-466c-8b35-fe6f7d5149b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:30.492820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1406/2848] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:46:30 2022] DELETE /identity/v3/projects/076e4d9f49a944c487077e845726ad82 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:46:30.497871 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a88d5-dabc-4b03-a206-3fae0f60d9f3 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:30.498153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a88d5-dabc-4b03-a206-3fae0f60d9f3 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:30.498428 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a88d5-dabc-4b03-a206-3fae0f60d9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:30.518927 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d14a88d5-dabc-4b03-a206-3fae0f60d9f3 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:46:30.520182 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d14a88d5-dabc-4b03-a206-3fae0f60d9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:30.587027 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1443/2849] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:46:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:46:30.595063 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16744273-9b37-4d73-afd1-43aedb42e577 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:30.595269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16744273-9b37-4d73-afd1-43aedb42e577 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:30.595516 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16744273-9b37-4d73-afd1-43aedb42e577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:30.615328 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-16744273-9b37-4d73-afd1-43aedb42e577 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:46:30.616641 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-16744273-9b37-4d73-afd1-43aedb42e577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:30.624518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1407/2850] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:46:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:46:30.628930 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a88d5-dabc-4b03-a206-3fae0f60d9f3 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:30.639339 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a88d5-dabc-4b03-a206-3fae0f60d9f3 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:30.644240 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce4685-fb77-447c-8e82-f7385ab9302a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:30.644855 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce4685-fb77-447c-8e82-f7385ab9302a admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:30.645143 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce4685-fb77-447c-8e82-f7385ab9302a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:30.646054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce4685-fb77-447c-8e82-f7385ab9302a admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:30.646843 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce4685-fb77-447c-8e82-f7385ab9302a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:46:30.647346 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce4685-fb77-447c-8e82-f7385ab9302a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:30.649046 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce4685-fb77-447c-8e82-f7385ab9302a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:30.661033 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1444/2851] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:46:30.666531 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16744273-9b37-4d73-afd1-43aedb42e577 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:30.675848 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16744273-9b37-4d73-afd1-43aedb42e577 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:30.681354 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2deeeb4-bf9a-4a20-a6f6-086ebc168412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:30.682014 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2deeeb4-bf9a-4a20-a6f6-086ebc168412 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:30.682353 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2deeeb4-bf9a-4a20-a6f6-086ebc168412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:30.682672 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2deeeb4-bf9a-4a20-a6f6-086ebc168412 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:30.683339 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2deeeb4-bf9a-4a20-a6f6-086ebc168412 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:30.685415 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2deeeb4-bf9a-4a20-a6f6-086ebc168412 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:30.725079 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1408/2852] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:46:30 2022] POST /identity/v3/projects => generated 369 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:46:30.729422 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce4685-fb77-447c-8e82-f7385ab9302a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:30.741373 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce4685-fb77-447c-8e82-f7385ab9302a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:30.746801 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:46:30.803852 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998d591-212f-4926-8e84-5fe08a0888c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:30.804572 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998d591-212f-4926-8e84-5fe08a0888c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:30.804873 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998d591-212f-4926-8e84-5fe08a0888c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:30.805270 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998d591-212f-4926-8e84-5fe08a0888c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:30.805924 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998d591-212f-4926-8e84-5fe08a0888c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:30.807842 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998d591-212f-4926-8e84-5fe08a0888c7 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:30.831113 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1445/2853] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:46:30 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:46:30.835344 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2deeeb4-bf9a-4a20-a6f6-086ebc168412 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:30.845508 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2deeeb4-bf9a-4a20-a6f6-086ebc168412 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:30.850193 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b430a-f2d4-4aa2-869b-68953c9e95c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:30.850795 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b430a-f2d4-4aa2-869b-68953c9e95c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:30.851068 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b430a-f2d4-4aa2-869b-68953c9e95c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:30.851336 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b430a-f2d4-4aa2-869b-68953c9e95c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:30.852001 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b430a-f2d4-4aa2-869b-68953c9e95c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:30.853622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b430a-f2d4-4aa2-869b-68953c9e95c3 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:30.865913 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1409/2854] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:46:30.870076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998d591-212f-4926-8e84-5fe08a0888c7 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:30.880983 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998d591-212f-4926-8e84-5fe08a0888c7 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:30.885596 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5b725-2d74-4c94-9c34-71cf185b5d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:30.886204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5b725-2d74-4c94-9c34-71cf185b5d17 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:30.886485 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5b725-2d74-4c94-9c34-71cf185b5d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:30.886760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5b725-2d74-4c94-9c34-71cf185b5d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:30.887407 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5b725-2d74-4c94-9c34-71cf185b5d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:30.889034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5b725-2d74-4c94-9c34-71cf185b5d17 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:30.899687 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1446/2855] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:46:30.903949 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b430a-f2d4-4aa2-869b-68953c9e95c3 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:30.914182 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b430a-f2d4-4aa2-869b-68953c9e95c3 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:30.918727 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd27ca-7bdb-4104-8aff-47ef6c42b066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:30.919372 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd27ca-7bdb-4104-8aff-47ef6c42b066 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:30.919646 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd27ca-7bdb-4104-8aff-47ef6c42b066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:30.919914 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd27ca-7bdb-4104-8aff-47ef6c42b066 admin admin] PATH_INFO: `/v3/projects/7b4da4cff9234d9da0fffe1a293290e0/users/26ad620c1b80494f89b8e3b6b2dc3606/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:30.935118 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd27ca-7bdb-4104-8aff-47ef6c42b066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b4da4cff9234d9da0fffe1a293290e0, user_id=26ad620c1b80494f89b8e3b6b2dc3606, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:30.937786 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd27ca-7bdb-4104-8aff-47ef6c42b066 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:30.951668 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1410/2856] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:46:30 2022] PUT /identity/v3/projects/7b4da4cff9234d9da0fffe1a293290e0/users/26ad620c1b80494f89b8e3b6b2dc3606/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:46:30.963090 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f4457-61b1-4e86-9976-5378ff3177a2 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:30.963598 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f4457-61b1-4e86-9976-5378ff3177a2 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:30.963732 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f4457-61b1-4e86-9976-5378ff3177a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:30.992969 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-d50f4457-61b1-4e86-9976-5378ff3177a2 None None] MFA Rules not processed for user `26ad620c1b80494f89b8e3b6b2dc3606`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:46:30.994199 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-d50f4457-61b1-4e86-9976-5378ff3177a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:31.057568 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1447/2857] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:46:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:46:31.062035 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd27ca-7bdb-4104-8aff-47ef6c42b066 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:31.068591 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:46:31.134908 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd27ca-7bdb-4104-8aff-47ef6c42b066 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:31.140281 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577600f-0902-4623-816f-53c4a7b46586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:31.140894 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577600f-0902-4623-816f-53c4a7b46586 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:31.141258 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577600f-0902-4623-816f-53c4a7b46586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:31.141593 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577600f-0902-4623-816f-53c4a7b46586 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:31.142466 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577600f-0902-4623-816f-53c4a7b46586 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:31.144427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577600f-0902-4623-816f-53c4a7b46586 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:31.172324 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1411/2858] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:46:31 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:46:31.176517 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f4457-61b1-4e86-9976-5378ff3177a2 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:31.187961 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f4457-61b1-4e86-9976-5378ff3177a2 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:31.244930 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6be8d-905f-43d7-a8ff-412ab4f6ddcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:31.245539 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6be8d-905f-43d7-a8ff-412ab4f6ddcd admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:31.245816 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6be8d-905f-43d7-a8ff-412ab4f6ddcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:31.246086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6be8d-905f-43d7-a8ff-412ab4f6ddcd admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:31.246724 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e6be8d-905f-43d7-a8ff-412ab4f6ddcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:31.248655 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e6be8d-905f-43d7-a8ff-412ab4f6ddcd admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:31.269500 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1448/2859] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:46:31 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:46:31.273653 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577600f-0902-4623-816f-53c4a7b46586 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:31.283729 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577600f-0902-4623-816f-53c4a7b46586 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:31.290481 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065280b-2c36-43c5-a0e2-efb6e9ceeff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:31.291134 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065280b-2c36-43c5-a0e2-efb6e9ceeff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:31.291409 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065280b-2c36-43c5-a0e2-efb6e9ceeff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:31.291675 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065280b-2c36-43c5-a0e2-efb6e9ceeff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:31.292343 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065280b-2c36-43c5-a0e2-efb6e9ceeff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:31.293923 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4065280b-2c36-43c5-a0e2-efb6e9ceeff7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:31.305706 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1412/2860] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:46:31.309938 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6be8d-905f-43d7-a8ff-412ab4f6ddcd admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:31.316365 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:46:31.322598 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6be8d-905f-43d7-a8ff-412ab4f6ddcd admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:31.328614 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd7bb0-517e-427f-ab44-4da75dbee492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:31.329306 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd7bb0-517e-427f-ab44-4da75dbee492 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:31.329584 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd7bb0-517e-427f-ab44-4da75dbee492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:31.329856 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd7bb0-517e-427f-ab44-4da75dbee492 admin admin] PATH_INFO: `/v3/projects/f218663ffb4d463f928f1dbdb250bfeb/users/ac5f9b56450846069c9658d29a4cc2c8/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:31.349839 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd7bb0-517e-427f-ab44-4da75dbee492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f218663ffb4d463f928f1dbdb250bfeb, user_id=ac5f9b56450846069c9658d29a4cc2c8, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:31.352723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd7bb0-517e-427f-ab44-4da75dbee492 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:31.367663 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1449/2861] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:46:31 2022] PUT /identity/v3/projects/f218663ffb4d463f928f1dbdb250bfeb/users/ac5f9b56450846069c9658d29a4cc2c8/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:46:31.371648 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065280b-2c36-43c5-a0e2-efb6e9ceeff7 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:31.382493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065280b-2c36-43c5-a0e2-efb6e9ceeff7 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:31.440783 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87225bb5-edb7-4d3a-88c8-931a696927df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:31.441390 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87225bb5-edb7-4d3a-88c8-931a696927df admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:31.441671 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87225bb5-edb7-4d3a-88c8-931a696927df admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:31.441939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87225bb5-edb7-4d3a-88c8-931a696927df admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:31.442874 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87225bb5-edb7-4d3a-88c8-931a696927df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:31.444493 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87225bb5-edb7-4d3a-88c8-931a696927df admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:31.454739 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1413/2862] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:46:31.459101 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd7bb0-517e-427f-ab44-4da75dbee492 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:31.470303 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd7bb0-517e-427f-ab44-4da75dbee492 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:31.475502 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20db5c-c09c-4eaa-aed0-414036267ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:31.476197 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20db5c-c09c-4eaa-aed0-414036267ab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:31.476482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20db5c-c09c-4eaa-aed0-414036267ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:31.476760 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20db5c-c09c-4eaa-aed0-414036267ab7 admin admin] PATH_INFO: `/v3/projects/f218663ffb4d463f928f1dbdb250bfeb/users/ac5f9b56450846069c9658d29a4cc2c8/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:31.478707 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e20db5c-c09c-4eaa-aed0-414036267ab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f218663ffb4d463f928f1dbdb250bfeb, user_id=ac5f9b56450846069c9658d29a4cc2c8, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:31.481450 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e20db5c-c09c-4eaa-aed0-414036267ab7 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:31.493148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1450/2863] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:46:31 2022] PUT /identity/v3/projects/f218663ffb4d463f928f1dbdb250bfeb/users/ac5f9b56450846069c9658d29a4cc2c8/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:46:31.503567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32244e0-bd04-4bc2-936e-bffa8cb826e7 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:31.503834 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32244e0-bd04-4bc2-936e-bffa8cb826e7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:31.504145 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32244e0-bd04-4bc2-936e-bffa8cb826e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:31.532932 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-a32244e0-bd04-4bc2-936e-bffa8cb826e7 None None] MFA Rules not processed for user `ac5f9b56450846069c9658d29a4cc2c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:46:31.534137 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a32244e0-bd04-4bc2-936e-bffa8cb826e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:31.598263 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:46:31.606115 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1414/2864] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:46:31 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:46:31.617806 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20db5c-c09c-4eaa-aed0-414036267ab7 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:31.619550 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-3e20db5c-c09c-4eaa-aed0-414036267ab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:31.677592 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20db5c-c09c-4eaa-aed0-414036267ab7 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:31.681320 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011823b-11c7-4c44-9332-5aab1e353460 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:31.681941 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011823b-11c7-4c44-9332-5aab1e353460 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:31.682224 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011823b-11c7-4c44-9332-5aab1e353460 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:31.682497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011823b-11c7-4c44-9332-5aab1e353460 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:31.691721 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011823b-11c7-4c44-9332-5aab1e353460 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:31.693482 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011823b-11c7-4c44-9332-5aab1e353460 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:31.705712 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1451/2865] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:46:31 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:46:31.752569 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32244e0-bd04-4bc2-936e-bffa8cb826e7 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:31.753951 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-a32244e0-bd04-4bc2-936e-bffa8cb826e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:31.766276 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32244e0-bd04-4bc2-936e-bffa8cb826e7 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:31.771116 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8617a-0a9d-48d8-b33e-e982b41e3ab9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:31.771730 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8617a-0a9d-48d8-b33e-e982b41e3ab9 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:31.772047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8617a-0a9d-48d8-b33e-e982b41e3ab9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:31.772326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8617a-0a9d-48d8-b33e-e982b41e3ab9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:31.844987 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8617a-0a9d-48d8-b33e-e982b41e3ab9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:31.849086 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8617a-0a9d-48d8-b33e-e982b41e3ab9 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:31.864145 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1415/2866] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:46:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 29 20:46:35.343484 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011823b-11c7-4c44-9332-5aab1e353460 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:35.406871 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011823b-11c7-4c44-9332-5aab1e353460 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:35.411838 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7414c5a-be46-4913-8cf4-cd56bb9259f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:35.412504 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7414c5a-be46-4913-8cf4-cd56bb9259f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:35.412783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7414c5a-be46-4913-8cf4-cd56bb9259f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:35.413055 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7414c5a-be46-4913-8cf4-cd56bb9259f8 admin admin] PATH_INFO: `/v3/users/26ad620c1b80494f89b8e3b6b2dc3606` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:35.414162 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7414c5a-be46-4913-8cf4-cd56bb9259f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26ad620c1b80494f89b8e3b6b2dc3606)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:35.416107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7414c5a-be46-4913-8cf4-cd56bb9259f8 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:35.467124 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e7414c5a-be46-4913-8cf4-cd56bb9259f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26ad620c1b80494f89b8e3b6b2dc3606'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:35.474914 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e7414c5a-be46-4913-8cf4-cd56bb9259f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26ad620c1b80494f89b8e3b6b2dc3606'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:35.482335 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e7414c5a-be46-4913-8cf4-cd56bb9259f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26ad620c1b80494f89b8e3b6b2dc3606'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:35.483061 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e7414c5a-be46-4913-8cf4-cd56bb9259f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26ad620c1b80494f89b8e3b6b2dc3606'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:35.483695 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e7414c5a-be46-4913-8cf4-cd56bb9259f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26ad620c1b80494f89b8e3b6b2dc3606'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:35.492479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:46:35.507150 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1452/2867] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:46:35 2022] DELETE /identity/v3/users/26ad620c1b80494f89b8e3b6b2dc3606 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:46:35.512563 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d7889-ce8c-4ebd-98f2-bd10349e348f None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:35.513135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d7889-ce8c-4ebd-98f2-bd10349e348f None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:35.513135 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d7889-ce8c-4ebd-98f2-bd10349e348f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:35.534262 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-547d7889-ce8c-4ebd-98f2-bd10349e348f None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:46:35.535627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-547d7889-ce8c-4ebd-98f2-bd10349e348f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:35.597302 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1416/2868] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:46:35.609191 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7414c5a-be46-4913-8cf4-cd56bb9259f8 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:35.610640 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-e7414c5a-be46-4913-8cf4-cd56bb9259f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:35.685853 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7414c5a-be46-4913-8cf4-cd56bb9259f8 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:35.690007 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83476eee-8351-4823-8c53-a7bb5abdb0b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:35.690740 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83476eee-8351-4823-8c53-a7bb5abdb0b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:35.691125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83476eee-8351-4823-8c53-a7bb5abdb0b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:35.691525 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83476eee-8351-4823-8c53-a7bb5abdb0b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:35.702447 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83476eee-8351-4823-8c53-a7bb5abdb0b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:35.703891 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83476eee-8351-4823-8c53-a7bb5abdb0b6 service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:35.716691 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1453/2869] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:46:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:46:36.028605 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d7889-ce8c-4ebd-98f2-bd10349e348f None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:36.030515 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-547d7889-ce8c-4ebd-98f2-bd10349e348f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:36.049045 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d7889-ce8c-4ebd-98f2-bd10349e348f None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:36.055844 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83beb7-c1c5-41db-afd0-df64008974eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:36.056769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83beb7-c1c5-41db-afd0-df64008974eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:36.056973 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83beb7-c1c5-41db-afd0-df64008974eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:36.057163 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83beb7-c1c5-41db-afd0-df64008974eb admin admin] PATH_INFO: `/v3/projects/7b4da4cff9234d9da0fffe1a293290e0` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:36.058393 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83beb7-c1c5-41db-afd0-df64008974eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b4da4cff9234d9da0fffe1a293290e0)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:36.060485 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83beb7-c1c5-41db-afd0-df64008974eb admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:36.112117 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5f83beb7-c1c5-41db-afd0-df64008974eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b4da4cff9234d9da0fffe1a293290e0', 'request_id': 'req-5f83beb7-c1c5-41db-afd0-df64008974eb'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:36.117555 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5f83beb7-c1c5-41db-afd0-df64008974eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b4da4cff9234d9da0fffe1a293290e0', 'request_id': 'req-5f83beb7-c1c5-41db-afd0-df64008974eb'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:36.125615 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5f83beb7-c1c5-41db-afd0-df64008974eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b4da4cff9234d9da0fffe1a293290e0', 'request_id': 'req-5f83beb7-c1c5-41db-afd0-df64008974eb'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:36.137793 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5f83beb7-c1c5-41db-afd0-df64008974eb admin admin] The token cache is being invalidate because project 7b4da4cff9234d9da0fffe1a293290e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:46:36.138100 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5f83beb7-c1c5-41db-afd0-df64008974eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:36.138761 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-5f83beb7-c1c5-41db-afd0-df64008974eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:36.140583 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1417/2870] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:46:36 2022] DELETE /identity/v3/projects/7b4da4cff9234d9da0fffe1a293290e0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:46:36.146991 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83476eee-8351-4823-8c53-a7bb5abdb0b6 service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:36.157034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83476eee-8351-4823-8c53-a7bb5abdb0b6 service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:36.226717 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c679c-d743-4e12-a659-c975e59009c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:36.227669 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c679c-d743-4e12-a659-c975e59009c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:36.227991 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c679c-d743-4e12-a659-c975e59009c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:36.228280 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c679c-d743-4e12-a659-c975e59009c9 admin admin] PATH_INFO: `/v3/users/ac5f9b56450846069c9658d29a4cc2c8` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:36.229579 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c679c-d743-4e12-a659-c975e59009c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac5f9b56450846069c9658d29a4cc2c8)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:36.232076 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c679c-d743-4e12-a659-c975e59009c9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:36.301444 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-929c679c-d743-4e12-a659-c975e59009c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac5f9b56450846069c9658d29a4cc2c8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:36.311786 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-929c679c-d743-4e12-a659-c975e59009c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac5f9b56450846069c9658d29a4cc2c8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:36.323192 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-929c679c-d743-4e12-a659-c975e59009c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac5f9b56450846069c9658d29a4cc2c8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:36.324234 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-929c679c-d743-4e12-a659-c975e59009c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac5f9b56450846069c9658d29a4cc2c8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:36.325119 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-929c679c-d743-4e12-a659-c975e59009c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac5f9b56450846069c9658d29a4cc2c8'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:36.352025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1454/2871] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:46:36 2022] DELETE /identity/v3/users/ac5f9b56450846069c9658d29a4cc2c8 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:46:36.646310 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83beb7-c1c5-41db-afd0-df64008974eb admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:36.647911 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-5f83beb7-c1c5-41db-afd0-df64008974eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:36.663491 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83beb7-c1c5-41db-afd0-df64008974eb admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:36.686747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:46:36.735254 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9056b-5d3c-463f-9aee-d0636191a3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:36.735897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9056b-5d3c-463f-9aee-d0636191a3b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:36.736206 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9056b-5d3c-463f-9aee-d0636191a3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:36.736482 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9056b-5d3c-463f-9aee-d0636191a3b6 admin admin] PATH_INFO: `/v3/projects/f218663ffb4d463f928f1dbdb250bfeb` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:36.737611 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e9056b-5d3c-463f-9aee-d0636191a3b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f218663ffb4d463f928f1dbdb250bfeb)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:36.740143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e9056b-5d3c-463f-9aee-d0636191a3b6 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:36.789262 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-25e9056b-5d3c-463f-9aee-d0636191a3b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f218663ffb4d463f928f1dbdb250bfeb', 'request_id': 'req-25e9056b-5d3c-463f-9aee-d0636191a3b6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:36.798782 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-25e9056b-5d3c-463f-9aee-d0636191a3b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f218663ffb4d463f928f1dbdb250bfeb', 'request_id': 'req-25e9056b-5d3c-463f-9aee-d0636191a3b6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:36.808962 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-25e9056b-5d3c-463f-9aee-d0636191a3b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f218663ffb4d463f928f1dbdb250bfeb', 'request_id': 'req-25e9056b-5d3c-463f-9aee-d0636191a3b6'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:36.820824 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-25e9056b-5d3c-463f-9aee-d0636191a3b6 admin admin] The token cache is being invalidate because project f218663ffb4d463f928f1dbdb250bfeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:46:36.821143 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-25e9056b-5d3c-463f-9aee-d0636191a3b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:36.821854 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-25e9056b-5d3c-463f-9aee-d0636191a3b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:36.823433 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1418/2872] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:46:36 2022] DELETE /identity/v3/projects/f218663ffb4d463f928f1dbdb250bfeb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:46:36.828637 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b922f8-0f2c-4c4b-9c76-54425594be4f None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:36.829117 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b922f8-0f2c-4c4b-9c76-54425594be4f None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:36.829516 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b922f8-0f2c-4c4b-9c76-54425594be4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:36.851013 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-46b922f8-0f2c-4c4b-9c76-54425594be4f None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:46:36.852271 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-46b922f8-0f2c-4c4b-9c76-54425594be4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:36.857967 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:46:36.918735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1455/2873] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:46:36.927627 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930ca0d-6245-4e73-af1f-1d1791b02258 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:36.928057 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930ca0d-6245-4e73-af1f-1d1791b02258 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:36.928338 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930ca0d-6245-4e73-af1f-1d1791b02258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:36.949201 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-e930ca0d-6245-4e73-af1f-1d1791b02258 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:46:36.950422 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e930ca0d-6245-4e73-af1f-1d1791b02258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:36.960184 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1419/2874] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:46:36.965464 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b922f8-0f2c-4c4b-9c76-54425594be4f None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:36.975895 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b922f8-0f2c-4c4b-9c76-54425594be4f None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:36.980702 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e0720-3087-4107-b63b-3d22d49cecdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:36.981317 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e0720-3087-4107-b63b-3d22d49cecdc admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:36.981598 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e0720-3087-4107-b63b-3d22d49cecdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:36.981874 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e0720-3087-4107-b63b-3d22d49cecdc admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:36.982529 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e0720-3087-4107-b63b-3d22d49cecdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:46:36.982835 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e0720-3087-4107-b63b-3d22d49cecdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:36.984457 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e0720-3087-4107-b63b-3d22d49cecdc admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:36.997748 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1456/2875] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:46:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:46:37.002427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930ca0d-6245-4e73-af1f-1d1791b02258 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:37.012103 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930ca0d-6245-4e73-af1f-1d1791b02258 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:37.017016 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf10a1-d286-40e9-ac01-a8c00b530676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:37.017621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf10a1-d286-40e9-ac01-a8c00b530676 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:37.017960 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf10a1-d286-40e9-ac01-a8c00b530676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:37.018240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf10a1-d286-40e9-ac01-a8c00b530676 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:37.018870 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf10a1-d286-40e9-ac01-a8c00b530676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:37.021340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf10a1-d286-40e9-ac01-a8c00b530676 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:37.056382 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1420/2876] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:46:37 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:46:37.061059 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e0720-3087-4107-b63b-3d22d49cecdc admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:37.071766 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e0720-3087-4107-b63b-3d22d49cecdc admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:37.133265 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0c1b2-d7d5-4a9c-a144-f62efbb06410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:37.134246 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0c1b2-d7d5-4a9c-a144-f62efbb06410 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:37.134715 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0c1b2-d7d5-4a9c-a144-f62efbb06410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:37.135189 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0c1b2-d7d5-4a9c-a144-f62efbb06410 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:37.136374 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0c1b2-d7d5-4a9c-a144-f62efbb06410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:37.139901 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0c1b2-d7d5-4a9c-a144-f62efbb06410 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:37.171573 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1457/2877] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:46:37 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:46:37.174820 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf10a1-d286-40e9-ac01-a8c00b530676 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:37.186064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf10a1-d286-40e9-ac01-a8c00b530676 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:37.190886 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41396ca-9f38-4fca-9ee3-6fa48db6000b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:37.191497 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41396ca-9f38-4fca-9ee3-6fa48db6000b admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:37.191779 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41396ca-9f38-4fca-9ee3-6fa48db6000b admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:37.192833 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41396ca-9f38-4fca-9ee3-6fa48db6000b admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:37.193484 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41396ca-9f38-4fca-9ee3-6fa48db6000b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:37.195099 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41396ca-9f38-4fca-9ee3-6fa48db6000b admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:37.205918 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1421/2878] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:46:37.210751 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0c1b2-d7d5-4a9c-a144-f62efbb06410 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:37.221096 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0c1b2-d7d5-4a9c-a144-f62efbb06410 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:37.226452 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece4397d-dc55-4538-b733-4341c3a9a85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:37.227070 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4397d-dc55-4538-b733-4341c3a9a85b admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:37.227351 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4397d-dc55-4538-b733-4341c3a9a85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:37.227624 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece4397d-dc55-4538-b733-4341c3a9a85b admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:37.228306 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4397d-dc55-4538-b733-4341c3a9a85b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:37.229892 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece4397d-dc55-4538-b733-4341c3a9a85b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:37.241150 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1458/2879] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:46:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:46:37.245560 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41396ca-9f38-4fca-9ee3-6fa48db6000b admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:37.256187 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41396ca-9f38-4fca-9ee3-6fa48db6000b admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:37.260830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98518477-8fa0-49b1-98cd-62a07af9caf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:37.261494 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98518477-8fa0-49b1-98cd-62a07af9caf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:37.261773 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98518477-8fa0-49b1-98cd-62a07af9caf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:37.262046 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98518477-8fa0-49b1-98cd-62a07af9caf8 admin admin] PATH_INFO: `/v3/projects/b44b4c8fd6314b89b8ce9ffeaffa3805/users/98cc7c1b4efc493a80683d5d47b20ff4/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:37.277988 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98518477-8fa0-49b1-98cd-62a07af9caf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b44b4c8fd6314b89b8ce9ffeaffa3805, user_id=98cc7c1b4efc493a80683d5d47b20ff4, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:37.280700 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98518477-8fa0-49b1-98cd-62a07af9caf8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:37.295410 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1422/2880] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:46:37 2022] PUT /identity/v3/projects/b44b4c8fd6314b89b8ce9ffeaffa3805/users/98cc7c1b4efc493a80683d5d47b20ff4/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:46:37.300620 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b23fef-8862-46f3-bb77-cfb73572fb72 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:37.300910 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b23fef-8862-46f3-bb77-cfb73572fb72 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:37.301219 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b23fef-8862-46f3-bb77-cfb73572fb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:37.321997 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-c7b23fef-8862-46f3-bb77-cfb73572fb72 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:46:37.323229 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c7b23fef-8862-46f3-bb77-cfb73572fb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:37.388629 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1459/2881] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:46:37.399983 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98518477-8fa0-49b1-98cd-62a07af9caf8 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:37.401716 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-98518477-8fa0-49b1-98cd-62a07af9caf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:37.459831 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:46:37.464451 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98518477-8fa0-49b1-98cd-62a07af9caf8 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:37.469552 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b549ccd-0ed5-4324-8977-ecbc830cca96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:37.470604 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b549ccd-0ed5-4324-8977-ecbc830cca96 service neutron] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:37.471036 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b549ccd-0ed5-4324-8977-ecbc830cca96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:37.471407 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b549ccd-0ed5-4324-8977-ecbc830cca96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:37.481633 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b549ccd-0ed5-4324-8977-ecbc830cca96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:37.483058 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b549ccd-0ed5-4324-8977-ecbc830cca96 service neutron] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:37.494057 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1423/2882] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:46:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:46:38.353329 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5f1fa-df90-4aa2-aff9-4ef0b9a3e642 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:38.353944 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5f1fa-df90-4aa2-aff9-4ef0b9a3e642 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:38.354417 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5f1fa-df90-4aa2-aff9-4ef0b9a3e642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:38.392971 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-45b5f1fa-df90-4aa2-aff9-4ef0b9a3e642 None None] MFA Rules not processed for user `98cc7c1b4efc493a80683d5d47b20ff4`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:46:38.394801 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-45b5f1fa-df90-4aa2-aff9-4ef0b9a3e642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:38.405955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:46:38.497788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1460/2883] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:46:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 29 20:46:38.502876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b549ccd-0ed5-4324-8977-ecbc830cca96 service neutron] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:38.516332 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b549ccd-0ed5-4324-8977-ecbc830cca96 service neutron] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:38.521707 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2fdf8-c800-4d8b-86c5-ef04dbb2b3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:38.522801 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2fdf8-c800-4d8b-86c5-ef04dbb2b3dc admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:38.522801 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2fdf8-c800-4d8b-86c5-ef04dbb2b3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:38.523052 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac2fdf8-c800-4d8b-86c5-ef04dbb2b3dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:38.523465 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac2fdf8-c800-4d8b-86c5-ef04dbb2b3dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:38.525530 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac2fdf8-c800-4d8b-86c5-ef04dbb2b3dc admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:38.559411 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1424/2884] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:46:38 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:46:38.563574 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5f1fa-df90-4aa2-aff9-4ef0b9a3e642 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:38.575496 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5f1fa-df90-4aa2-aff9-4ef0b9a3e642 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:38.650963 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542d0c0-1fcb-4974-97c2-8333617e5573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:38.651534 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542d0c0-1fcb-4974-97c2-8333617e5573 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:38.651863 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542d0c0-1fcb-4974-97c2-8333617e5573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:38.652198 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542d0c0-1fcb-4974-97c2-8333617e5573 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:38.653135 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2542d0c0-1fcb-4974-97c2-8333617e5573 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:38.655386 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2542d0c0-1fcb-4974-97c2-8333617e5573 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:38.684116 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1461/2885] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:46:38 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:46:38.688700 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2fdf8-c800-4d8b-86c5-ef04dbb2b3dc admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:38.700091 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac2fdf8-c800-4d8b-86c5-ef04dbb2b3dc admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:38.704887 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe368b-b699-43a6-bb4c-d6168bff9317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:38.705553 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe368b-b699-43a6-bb4c-d6168bff9317 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:38.705830 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe368b-b699-43a6-bb4c-d6168bff9317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:38.706098 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe368b-b699-43a6-bb4c-d6168bff9317 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:38.706833 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe368b-b699-43a6-bb4c-d6168bff9317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:38.709118 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe368b-b699-43a6-bb4c-d6168bff9317 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:38.721856 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1425/2886] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:46:38.726513 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542d0c0-1fcb-4974-97c2-8333617e5573 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:38.738107 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542d0c0-1fcb-4974-97c2-8333617e5573 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:38.743650 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0871703-a973-4ac6-bfd4-8e3c08e1e5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:38.744369 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0871703-a973-4ac6-bfd4-8e3c08e1e5d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:38.744668 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0871703-a973-4ac6-bfd4-8e3c08e1e5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:38.744937 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0871703-a973-4ac6-bfd4-8e3c08e1e5d9 admin admin] PATH_INFO: `/v3/projects/974ea495ba284902993d336b012efd4a/users/70a20c87dba74296b90de4ab63957a8d/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:38.763085 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0871703-a973-4ac6-bfd4-8e3c08e1e5d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=974ea495ba284902993d336b012efd4a, user_id=70a20c87dba74296b90de4ab63957a8d, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:38.766576 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0871703-a973-4ac6-bfd4-8e3c08e1e5d9 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:38.791094 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1462/2887] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:46:38 2022] PUT /identity/v3/projects/974ea495ba284902993d336b012efd4a/users/70a20c87dba74296b90de4ab63957a8d/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:46:38.794210 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe368b-b699-43a6-bb4c-d6168bff9317 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:38.806256 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe368b-b699-43a6-bb4c-d6168bff9317 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:38.872133 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98f043-97a3-4377-b15b-fed5ff99755f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:38.872738 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98f043-97a3-4377-b15b-fed5ff99755f admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:38.873018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98f043-97a3-4377-b15b-fed5ff99755f admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:38.873297 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98f043-97a3-4377-b15b-fed5ff99755f admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:38.873941 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98f043-97a3-4377-b15b-fed5ff99755f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:38.875527 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98f043-97a3-4377-b15b-fed5ff99755f admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:38.888939 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1426/2888] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:46:38.893739 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0871703-a973-4ac6-bfd4-8e3c08e1e5d9 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:38.906408 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0871703-a973-4ac6-bfd4-8e3c08e1e5d9 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:38.912255 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5688d2-3edc-496e-b5de-caca2d114f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:38.913010 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5688d2-3edc-496e-b5de-caca2d114f0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:38.913403 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5688d2-3edc-496e-b5de-caca2d114f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:38.913639 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5688d2-3edc-496e-b5de-caca2d114f0a admin admin] PATH_INFO: `/v3/projects/974ea495ba284902993d336b012efd4a/users/70a20c87dba74296b90de4ab63957a8d/roles/322da7bd64024770a64bdb73e5c54719` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:38.915721 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5688d2-3edc-496e-b5de-caca2d114f0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=974ea495ba284902993d336b012efd4a, user_id=70a20c87dba74296b90de4ab63957a8d, role_id=322da7bd64024770a64bdb73e5c54719)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:38.918534 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5688d2-3edc-496e-b5de-caca2d114f0a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:38.933034 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1463/2889] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:46:38 2022] PUT /identity/v3/projects/974ea495ba284902993d336b012efd4a/users/70a20c87dba74296b90de4ab63957a8d/roles/322da7bd64024770a64bdb73e5c54719 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:46:39.718747 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10275975-bd05-4d38-83e9-8089a27cd40c None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:39.719300 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10275975-bd05-4d38-83e9-8089a27cd40c None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:39.719452 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10275975-bd05-4d38-83e9-8089a27cd40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:39.754110 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-10275975-bd05-4d38-83e9-8089a27cd40c None None] MFA Rules not processed for user `70a20c87dba74296b90de4ab63957a8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:46:39.756096 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-10275975-bd05-4d38-83e9-8089a27cd40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:39.851177 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1427/2890] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:46:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:46:39.874883 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5688d2-3edc-496e-b5de-caca2d114f0a admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:39.877167 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-2a5688d2-3edc-496e-b5de-caca2d114f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:39.974148 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5688d2-3edc-496e-b5de-caca2d114f0a admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:39.980055 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd90bbb-72e3-4346-9d01-8cd712d10446 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:39.980519 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd90bbb-72e3-4346-9d01-8cd712d10446 service cinder] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:39.980822 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd90bbb-72e3-4346-9d01-8cd712d10446 service cinder] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:39.981173 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd90bbb-72e3-4346-9d01-8cd712d10446 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:39.997390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:46:40.045951 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd90bbb-72e3-4346-9d01-8cd712d10446 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:40.047497 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd90bbb-72e3-4346-9d01-8cd712d10446 service cinder] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:40.058105 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1464/2891] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:46:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:46:41.499089 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10275975-bd05-4d38-83e9-8089a27cd40c None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:41.501275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-10275975-bd05-4d38-83e9-8089a27cd40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:41.538558 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10275975-bd05-4d38-83e9-8089a27cd40c None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:41.545388 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4168abf-d82a-426a-8fed-2d8438a1070b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:41.546363 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4168abf-d82a-426a-8fed-2d8438a1070b service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:41.546816 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4168abf-d82a-426a-8fed-2d8438a1070b service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:41.547267 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4168abf-d82a-426a-8fed-2d8438a1070b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:41.564208 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4168abf-d82a-426a-8fed-2d8438a1070b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:41.566721 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4168abf-d82a-426a-8fed-2d8438a1070b service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:41.584690 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1428/2892] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:46:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:46:50.053769 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=21 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:46:51.578116 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=87,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:46:55.135392 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd90bbb-72e3-4346-9d01-8cd712d10446 service cinder] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:55.211396 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd90bbb-72e3-4346-9d01-8cd712d10446 service cinder] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:55.216008 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028783c-5d3d-4bed-9592-f1d33f6de621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:55.216647 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028783c-5d3d-4bed-9592-f1d33f6de621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:55.216928 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028783c-5d3d-4bed-9592-f1d33f6de621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:55.217204 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028783c-5d3d-4bed-9592-f1d33f6de621 admin admin] PATH_INFO: `/v3/users/98cc7c1b4efc493a80683d5d47b20ff4` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:55.218312 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028783c-5d3d-4bed-9592-f1d33f6de621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98cc7c1b4efc493a80683d5d47b20ff4)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:55.220258 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028783c-5d3d-4bed-9592-f1d33f6de621 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:55.268755 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c028783c-5d3d-4bed-9592-f1d33f6de621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98cc7c1b4efc493a80683d5d47b20ff4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:55.275811 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c028783c-5d3d-4bed-9592-f1d33f6de621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98cc7c1b4efc493a80683d5d47b20ff4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:55.283318 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c028783c-5d3d-4bed-9592-f1d33f6de621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98cc7c1b4efc493a80683d5d47b20ff4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:55.284126 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c028783c-5d3d-4bed-9592-f1d33f6de621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98cc7c1b4efc493a80683d5d47b20ff4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:55.284844 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-c028783c-5d3d-4bed-9592-f1d33f6de621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98cc7c1b4efc493a80683d5d47b20ff4'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:55.309018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1465/2893] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:46:55 2022] DELETE /identity/v3/users/98cc7c1b4efc493a80683d5d47b20ff4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:46:55.530226 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4168abf-d82a-426a-8fed-2d8438a1070b service cinder] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:55.541920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4168abf-d82a-426a-8fed-2d8438a1070b service cinder] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:55.605948 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0099155-25a7-4fb6-8ad4-52edd8262ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:55.606594 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0099155-25a7-4fb6-8ad4-52edd8262ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:55.606877 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0099155-25a7-4fb6-8ad4-52edd8262ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:55.607154 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0099155-25a7-4fb6-8ad4-52edd8262ed4 admin admin] PATH_INFO: `/v3/projects/b44b4c8fd6314b89b8ce9ffeaffa3805` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:55.608312 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0099155-25a7-4fb6-8ad4-52edd8262ed4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b44b4c8fd6314b89b8ce9ffeaffa3805)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:55.610212 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0099155-25a7-4fb6-8ad4-52edd8262ed4 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:55.660576 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e0099155-25a7-4fb6-8ad4-52edd8262ed4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b44b4c8fd6314b89b8ce9ffeaffa3805', 'request_id': 'req-e0099155-25a7-4fb6-8ad4-52edd8262ed4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:55.666064 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e0099155-25a7-4fb6-8ad4-52edd8262ed4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b44b4c8fd6314b89b8ce9ffeaffa3805', 'request_id': 'req-e0099155-25a7-4fb6-8ad4-52edd8262ed4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:55.673286 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e0099155-25a7-4fb6-8ad4-52edd8262ed4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b44b4c8fd6314b89b8ce9ffeaffa3805', 'request_id': 'req-e0099155-25a7-4fb6-8ad4-52edd8262ed4'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:55.685759 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e0099155-25a7-4fb6-8ad4-52edd8262ed4 admin admin] The token cache is being invalidate because project b44b4c8fd6314b89b8ce9ffeaffa3805 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:46:55.686036 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e0099155-25a7-4fb6-8ad4-52edd8262ed4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:55.686671 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e0099155-25a7-4fb6-8ad4-52edd8262ed4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:55.688181 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1429/2894] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:46:55 2022] DELETE /identity/v3/projects/b44b4c8fd6314b89b8ce9ffeaffa3805 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:46:55.692184 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028783c-5d3d-4bed-9592-f1d33f6de621 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:55.693896 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c028783c-5d3d-4bed-9592-f1d33f6de621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:55.707706 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028783c-5d3d-4bed-9592-f1d33f6de621 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:55.767026 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6981baf-7b50-4a9b-ad2d-ff09c4818f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:55.767669 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6981baf-7b50-4a9b-ad2d-ff09c4818f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:55.768039 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6981baf-7b50-4a9b-ad2d-ff09c4818f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:55.768324 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6981baf-7b50-4a9b-ad2d-ff09c4818f9e admin admin] PATH_INFO: `/v3/users/70a20c87dba74296b90de4ab63957a8d` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:55.769456 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6981baf-7b50-4a9b-ad2d-ff09c4818f9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=70a20c87dba74296b90de4ab63957a8d)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:55.771472 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6981baf-7b50-4a9b-ad2d-ff09c4818f9e admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:55.792705 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:46:55.824856 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e6981baf-7b50-4a9b-ad2d-ff09c4818f9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70a20c87dba74296b90de4ab63957a8d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:55.832880 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e6981baf-7b50-4a9b-ad2d-ff09c4818f9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70a20c87dba74296b90de4ab63957a8d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:55.841360 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e6981baf-7b50-4a9b-ad2d-ff09c4818f9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70a20c87dba74296b90de4ab63957a8d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:55.842059 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e6981baf-7b50-4a9b-ad2d-ff09c4818f9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70a20c87dba74296b90de4ab63957a8d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:55.842830 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-e6981baf-7b50-4a9b-ad2d-ff09c4818f9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70a20c87dba74296b90de4ab63957a8d'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:55.861199 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1466/2895] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:46:55 2022] DELETE /identity/v3/users/70a20c87dba74296b90de4ab63957a8d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:46:56.058722 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0099155-25a7-4fb6-8ad4-52edd8262ed4 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:56.060233 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-e0099155-25a7-4fb6-8ad4-52edd8262ed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:56.074385 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0099155-25a7-4fb6-8ad4-52edd8262ed4 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:56.136630 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ae884-527f-452a-9f87-26836d7d1903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:56.137269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ae884-527f-452a-9f87-26836d7d1903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:56.137549 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ae884-527f-452a-9f87-26836d7d1903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:56.137825 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ae884-527f-452a-9f87-26836d7d1903 admin admin] PATH_INFO: `/v3/projects/974ea495ba284902993d336b012efd4a` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:56.138954 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ae884-527f-452a-9f87-26836d7d1903 admin admin] RBAC: Authorizing `identity:delete_project(project_id=974ea495ba284902993d336b012efd4a)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:56.141047 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ae884-527f-452a-9f87-26836d7d1903 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:56.180072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:46:56.194072 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e20ae884-527f-452a-9f87-26836d7d1903 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '974ea495ba284902993d336b012efd4a', 'request_id': 'req-e20ae884-527f-452a-9f87-26836d7d1903'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:56.199543 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e20ae884-527f-452a-9f87-26836d7d1903 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '974ea495ba284902993d336b012efd4a', 'request_id': 'req-e20ae884-527f-452a-9f87-26836d7d1903'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:56.207444 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e20ae884-527f-452a-9f87-26836d7d1903 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '974ea495ba284902993d336b012efd4a', 'request_id': 'req-e20ae884-527f-452a-9f87-26836d7d1903'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:56.222313 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e20ae884-527f-452a-9f87-26836d7d1903 admin admin] The token cache is being invalidate because project 974ea495ba284902993d336b012efd4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:46:56.222607 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e20ae884-527f-452a-9f87-26836d7d1903 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:56.223244 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-e20ae884-527f-452a-9f87-26836d7d1903 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:46:56.225597 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1430/2896] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:46:56 2022] DELETE /identity/v3/projects/974ea495ba284902993d336b012efd4a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:46:56.230744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaaba1-2017-4526-8bd2-0d1c4a2ce3e2 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:56.231313 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaaba1-2017-4526-8bd2-0d1c4a2ce3e2 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:56.231313 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaaba1-2017-4526-8bd2-0d1c4a2ce3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:56.251650 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-7cfaaba1-2017-4526-8bd2-0d1c4a2ce3e2 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:46:56.253059 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-7cfaaba1-2017-4526-8bd2-0d1c4a2ce3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:56.323777 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1467/2897] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:46:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:46:56.335210 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b688611-4b11-48b7-b9be-07e1e4d58ef6 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:56.335622 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b688611-4b11-48b7-b9be-07e1e4d58ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:56.336083 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b688611-4b11-48b7-b9be-07e1e4d58ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:56.356284 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-6b688611-4b11-48b7-b9be-07e1e4d58ef6 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:46:56.357518 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-6b688611-4b11-48b7-b9be-07e1e4d58ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:56.366057 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1431/2898] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:46:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:46:56.370562 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfaaba1-2017-4526-8bd2-0d1c4a2ce3e2 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:56.380665 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfaaba1-2017-4526-8bd2-0d1c4a2ce3e2 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:56.385302 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f31c31-ff2a-4f6d-a242-0e0ee6355183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:56.385987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f31c31-ff2a-4f6d-a242-0e0ee6355183 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:56.386176 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f31c31-ff2a-4f6d-a242-0e0ee6355183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:56.386446 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f31c31-ff2a-4f6d-a242-0e0ee6355183 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:56.387088 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f31c31-ff2a-4f6d-a242-0e0ee6355183 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:46:56.387389 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f31c31-ff2a-4f6d-a242-0e0ee6355183 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:56.388994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f31c31-ff2a-4f6d-a242-0e0ee6355183 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:56.402994 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1468/2899] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:46:56.407422 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b688611-4b11-48b7-b9be-07e1e4d58ef6 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:56.417689 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b688611-4b11-48b7-b9be-07e1e4d58ef6 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:56.422801 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e340e8a-604a-46e1-a360-6d3968c084b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:56.423419 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e340e8a-604a-46e1-a360-6d3968c084b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:56.423722 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e340e8a-604a-46e1-a360-6d3968c084b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:56.424030 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e340e8a-604a-46e1-a360-6d3968c084b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:56.424734 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e340e8a-604a-46e1-a360-6d3968c084b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:56.426799 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e340e8a-604a-46e1-a360-6d3968c084b7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:56.457234 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1432/2900] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:46:56 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:46:56.462434 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f31c31-ff2a-4f6d-a242-0e0ee6355183 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:56.472955 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f31c31-ff2a-4f6d-a242-0e0ee6355183 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:56.532174 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017aedf-2859-4b76-9864-2a602f8bd229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:56.532789 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017aedf-2859-4b76-9864-2a602f8bd229 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:56.533073 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017aedf-2859-4b76-9864-2a602f8bd229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:56.533348 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017aedf-2859-4b76-9864-2a602f8bd229 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:56.533979 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017aedf-2859-4b76-9864-2a602f8bd229 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:56.535918 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017aedf-2859-4b76-9864-2a602f8bd229 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:56.558078 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1469/2901] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:46:56 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:46:56.562340 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e340e8a-604a-46e1-a360-6d3968c084b7 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:56.576089 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e340e8a-604a-46e1-a360-6d3968c084b7 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:56.580876 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0a792-1e80-4184-be8d-15a481d4ccb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:56.581505 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0a792-1e80-4184-be8d-15a481d4ccb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:56.581788 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0a792-1e80-4184-be8d-15a481d4ccb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:56.582111 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0a792-1e80-4184-be8d-15a481d4ccb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:56.582731 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0a792-1e80-4184-be8d-15a481d4ccb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:56.584846 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0a792-1e80-4184-be8d-15a481d4ccb8 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:56.594999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1433/2902] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:46:56.599723 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017aedf-2859-4b76-9864-2a602f8bd229 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:56.610970 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017aedf-2859-4b76-9864-2a602f8bd229 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:56.616062 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3db509-b150-4c8f-accb-af2678a41c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:56.616601 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3db509-b150-4c8f-accb-af2678a41c42 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:56.616894 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3db509-b150-4c8f-accb-af2678a41c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:56.617173 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3db509-b150-4c8f-accb-af2678a41c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:56.617834 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3db509-b150-4c8f-accb-af2678a41c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:56.619440 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3db509-b150-4c8f-accb-af2678a41c42 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:56.631202 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1470/2903] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:46:56.635651 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0a792-1e80-4184-be8d-15a481d4ccb8 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:56.647141 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0a792-1e80-4184-be8d-15a481d4ccb8 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:56.652388 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027eb97d-13b3-48be-ad7a-2a84285a9e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:56.653081 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027eb97d-13b3-48be-ad7a-2a84285a9e05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:56.653360 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027eb97d-13b3-48be-ad7a-2a84285a9e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:56.653637 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027eb97d-13b3-48be-ad7a-2a84285a9e05 admin admin] PATH_INFO: `/v3/projects/f802bbea278e4cc5ad1cb62cbe400c53/users/0594f67a3468489f95bb3c9c9a98578e/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:56.669000 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027eb97d-13b3-48be-ad7a-2a84285a9e05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f802bbea278e4cc5ad1cb62cbe400c53, user_id=0594f67a3468489f95bb3c9c9a98578e, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:56.671630 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027eb97d-13b3-48be-ad7a-2a84285a9e05 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:56.693567 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1434/2904] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:46:56 2022] PUT /identity/v3/projects/f802bbea278e4cc5ad1cb62cbe400c53/users/0594f67a3468489f95bb3c9c9a98578e/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:46:56.705356 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7eb84-c466-42bf-915c-15aa1fde149e None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:56.705644 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7eb84-c466-42bf-915c-15aa1fde149e None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:56.705926 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7eb84-c466-42bf-915c-15aa1fde149e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:56.719026 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:46:56.743779 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-c2d7eb84-c466-42bf-915c-15aa1fde149e None None] MFA Rules not processed for user `0594f67a3468489f95bb3c9c9a98578e`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:46:56.745082 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-c2d7eb84-c466-42bf-915c-15aa1fde149e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:56.813018 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1471/2905] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:46:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:46:56.826407 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027eb97d-13b3-48be-ad7a-2a84285a9e05 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:46:56.828997 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-027eb97d-13b3-48be-ad7a-2a84285a9e05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:46:56.906320 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027eb97d-13b3-48be-ad7a-2a84285a9e05 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:46:56.909920 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad5ff4-a419-4e83-97ac-19d4ed8c5560 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d022ed8d2ac43ae816aa396a8e2f17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:46:56.910526 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad5ff4-a419-4e83-97ac-19d4ed8c5560 service glance] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:46:56.911801 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad5ff4-a419-4e83-97ac-19d4ed8c5560 service glance] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:46:56.911801 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad5ff4-a419-4e83-97ac-19d4ed8c5560 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:46:56.920990 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad5ff4-a419-4e83-97ac-19d4ed8c5560 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:46:56.922532 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad5ff4-a419-4e83-97ac-19d4ed8c5560 service glance] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:46:56.932231 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1435/2906] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:46:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:47:06.809932 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=30 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:47:06.927992 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=74,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:47:14.680078 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7eb84-c466-42bf-915c-15aa1fde149e None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:14.753873 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7eb84-c466-42bf-915c-15aa1fde149e None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:14.758518 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903075ba-6c83-426f-8414-ec8624f77ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:14.759435 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903075ba-6c83-426f-8414-ec8624f77ea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:14.759435 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903075ba-6c83-426f-8414-ec8624f77ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:14.760145 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903075ba-6c83-426f-8414-ec8624f77ea5 admin admin] PATH_INFO: `/v3/users/0594f67a3468489f95bb3c9c9a98578e` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:14.761231 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903075ba-6c83-426f-8414-ec8624f77ea5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0594f67a3468489f95bb3c9c9a98578e)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:14.763175 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903075ba-6c83-426f-8414-ec8624f77ea5 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:14.820251 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-903075ba-6c83-426f-8414-ec8624f77ea5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0594f67a3468489f95bb3c9c9a98578e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:14.827565 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-903075ba-6c83-426f-8414-ec8624f77ea5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0594f67a3468489f95bb3c9c9a98578e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:14.836537 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-903075ba-6c83-426f-8414-ec8624f77ea5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0594f67a3468489f95bb3c9c9a98578e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:14.837506 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-903075ba-6c83-426f-8414-ec8624f77ea5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0594f67a3468489f95bb3c9c9a98578e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:14.838547 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-903075ba-6c83-426f-8414-ec8624f77ea5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0594f67a3468489f95bb3c9c9a98578e'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:14.860997 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1472/2907] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:47:14 2022] DELETE /identity/v3/users/0594f67a3468489f95bb3c9c9a98578e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:47:14.865764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae182c-8d5f-4d6f-a595-1c276788dc9e None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:14.865764 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae182c-8d5f-4d6f-a595-1c276788dc9e None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:14.867512 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae182c-8d5f-4d6f-a595-1c276788dc9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:14.887186 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-d1ae182c-8d5f-4d6f-a595-1c276788dc9e None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:47:14.888446 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d1ae182c-8d5f-4d6f-a595-1c276788dc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:47:14.958021 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1436/2908] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:47:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:47:14.969788 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903075ba-6c83-426f-8414-ec8624f77ea5 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:14.971406 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-903075ba-6c83-426f-8414-ec8624f77ea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:47:15.035238 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903075ba-6c83-426f-8414-ec8624f77ea5 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:15.048104 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac618d-3872-4469-99e7-25f1ed774aad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:15.049079 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac618d-3872-4469-99e7-25f1ed774aad service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:15.049418 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac618d-3872-4469-99e7-25f1ed774aad service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:15.049736 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac618d-3872-4469-99e7-25f1ed774aad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:15.060736 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac618d-3872-4469-99e7-25f1ed774aad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:15.062162 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac618d-3872-4469-99e7-25f1ed774aad service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:15.072321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1473/2909] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:47:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:47:15.364603 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae182c-8d5f-4d6f-a595-1c276788dc9e None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:15.366067 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-d1ae182c-8d5f-4d6f-a595-1c276788dc9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:47:15.379251 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae182c-8d5f-4d6f-a595-1c276788dc9e None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:15.385508 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d50ae1-327d-44b2-81d3-65ecc9f864c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:15.389142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d50ae1-327d-44b2-81d3-65ecc9f864c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:15.389142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d50ae1-327d-44b2-81d3-65ecc9f864c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:15.389142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d50ae1-327d-44b2-81d3-65ecc9f864c7 admin admin] PATH_INFO: `/v3/projects/f802bbea278e4cc5ad1cb62cbe400c53` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:15.389142 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d50ae1-327d-44b2-81d3-65ecc9f864c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f802bbea278e4cc5ad1cb62cbe400c53)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:15.390806 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d50ae1-327d-44b2-81d3-65ecc9f864c7 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:15.456866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-96d50ae1-327d-44b2-81d3-65ecc9f864c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f802bbea278e4cc5ad1cb62cbe400c53', 'request_id': 'req-96d50ae1-327d-44b2-81d3-65ecc9f864c7'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:15.463499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-96d50ae1-327d-44b2-81d3-65ecc9f864c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f802bbea278e4cc5ad1cb62cbe400c53', 'request_id': 'req-96d50ae1-327d-44b2-81d3-65ecc9f864c7'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:15.472822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-96d50ae1-327d-44b2-81d3-65ecc9f864c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f802bbea278e4cc5ad1cb62cbe400c53', 'request_id': 'req-96d50ae1-327d-44b2-81d3-65ecc9f864c7'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:15.486174 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-96d50ae1-327d-44b2-81d3-65ecc9f864c7 admin admin] The token cache is being invalidate because project f802bbea278e4cc5ad1cb62cbe400c53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:47:15.486499 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-96d50ae1-327d-44b2-81d3-65ecc9f864c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:15.487097 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-96d50ae1-327d-44b2-81d3-65ecc9f864c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:15.488616 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1437/2910] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:47:15 2022] DELETE /identity/v3/projects/f802bbea278e4cc5ad1cb62cbe400c53 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:47:15.493885 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11b58e-105a-4ee4-ab86-259c1afc20ff None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:15.494205 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11b58e-105a-4ee4-ab86-259c1afc20ff None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:15.494518 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11b58e-105a-4ee4-ab86-259c1afc20ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:15.516769 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-4a11b58e-105a-4ee4-ab86-259c1afc20ff None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:47:15.518245 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4a11b58e-105a-4ee4-ab86-259c1afc20ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:47:15.538640 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:47:15.588698 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1474/2911] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:47:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:47:15.598548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ed6d8-9717-4fb5-b5c9-f444c4680f38 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:15.598822 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ed6d8-9717-4fb5-b5c9-f444c4680f38 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:15.599150 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ed6d8-9717-4fb5-b5c9-f444c4680f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:15.620848 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-fe8ed6d8-9717-4fb5-b5c9-f444c4680f38 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:47:15.622090 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-fe8ed6d8-9717-4fb5-b5c9-f444c4680f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:47:15.630778 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1438/2912] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:47:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:47:15.635649 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11b58e-105a-4ee4-ab86-259c1afc20ff None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:15.646988 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11b58e-105a-4ee4-ab86-259c1afc20ff None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:15.653146 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705670f2-a48d-4656-9a36-649bce66439b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:15.653798 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705670f2-a48d-4656-9a36-649bce66439b admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:15.654380 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705670f2-a48d-4656-9a36-649bce66439b admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:15.654668 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705670f2-a48d-4656-9a36-649bce66439b admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:15.655341 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705670f2-a48d-4656-9a36-649bce66439b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:47:15.655653 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705670f2-a48d-4656-9a36-649bce66439b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:15.657282 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705670f2-a48d-4656-9a36-649bce66439b admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:15.672025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1475/2913] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:47:15.676866 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ed6d8-9717-4fb5-b5c9-f444c4680f38 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:15.687650 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ed6d8-9717-4fb5-b5c9-f444c4680f38 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:15.693157 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56190f0f-ff53-4369-a914-ce0215ae5d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:15.693851 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56190f0f-ff53-4369-a914-ce0215ae5d67 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:15.694116 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56190f0f-ff53-4369-a914-ce0215ae5d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:15.694402 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56190f0f-ff53-4369-a914-ce0215ae5d67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:15.695845 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56190f0f-ff53-4369-a914-ce0215ae5d67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:15.698125 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56190f0f-ff53-4369-a914-ce0215ae5d67 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:15.728313 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1439/2914] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:47:15 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:47:15.732698 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705670f2-a48d-4656-9a36-649bce66439b admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:15.744023 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705670f2-a48d-4656-9a36-649bce66439b admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:15.806162 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cde770-34fb-4337-af3a-c61d2a6646e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:15.806818 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cde770-34fb-4337-af3a-c61d2a6646e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:15.807122 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cde770-34fb-4337-af3a-c61d2a6646e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:15.807413 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cde770-34fb-4337-af3a-c61d2a6646e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:15.808125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cde770-34fb-4337-af3a-c61d2a6646e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:15.810121 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cde770-34fb-4337-af3a-c61d2a6646e5 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:15.836501 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1476/2915] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:47:15 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:47:15.840681 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56190f0f-ff53-4369-a914-ce0215ae5d67 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:15.852155 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56190f0f-ff53-4369-a914-ce0215ae5d67 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:15.857023 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f668f-1381-4908-a08d-dd712b390908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:15.857621 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f668f-1381-4908-a08d-dd712b390908 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:15.857899 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f668f-1381-4908-a08d-dd712b390908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:15.858173 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f668f-1381-4908-a08d-dd712b390908 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:15.858811 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f668f-1381-4908-a08d-dd712b390908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:15.860446 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f668f-1381-4908-a08d-dd712b390908 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:15.871656 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1440/2916] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 29 20:47:15.875783 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cde770-34fb-4337-af3a-c61d2a6646e5 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:15.887098 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cde770-34fb-4337-af3a-c61d2a6646e5 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:15.891725 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d94b6d-4928-4184-b134-fc4e4d28f24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:15.892370 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d94b6d-4928-4184-b134-fc4e4d28f24a admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:15.892652 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d94b6d-4928-4184-b134-fc4e4d28f24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:15.892929 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d94b6d-4928-4184-b134-fc4e4d28f24a admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:15.893572 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d94b6d-4928-4184-b134-fc4e4d28f24a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:15.895153 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d94b6d-4928-4184-b134-fc4e4d28f24a admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:15.906019 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1477/2917] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:47:15.910275 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f668f-1381-4908-a08d-dd712b390908 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:15.920998 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f668f-1381-4908-a08d-dd712b390908 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:15.925700 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5cd28d-8f68-4e1b-84fe-f79b1ba5b16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:15.926356 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5cd28d-8f68-4e1b-84fe-f79b1ba5b16c admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:15.926638 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5cd28d-8f68-4e1b-84fe-f79b1ba5b16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:15.926912 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5cd28d-8f68-4e1b-84fe-f79b1ba5b16c admin admin] PATH_INFO: `/v3/projects/a1b9811d32784b20941b0ba3178568fb/users/39b3d9896a34489aa14cbb4210f6315a/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:15.943769 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5cd28d-8f68-4e1b-84fe-f79b1ba5b16c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1b9811d32784b20941b0ba3178568fb, user_id=39b3d9896a34489aa14cbb4210f6315a, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:15.947149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5cd28d-8f68-4e1b-84fe-f79b1ba5b16c admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:15.959814 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1441/2918] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:47:15 2022] PUT /identity/v3/projects/a1b9811d32784b20941b0ba3178568fb/users/39b3d9896a34489aa14cbb4210f6315a/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:47:15.970271 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec78d6-93ee-40ec-8b61-feae53cd5f16 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:15.970552 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec78d6-93ee-40ec-8b61-feae53cd5f16 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:15.970827 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec78d6-93ee-40ec-8b61-feae53cd5f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:16.004727 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-8bec78d6-93ee-40ec-8b61-feae53cd5f16 None None] MFA Rules not processed for user `39b3d9896a34489aa14cbb4210f6315a`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:47:16.005848 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-8bec78d6-93ee-40ec-8b61-feae53cd5f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:47:16.083964 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1478/2919] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:47:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:47:16.096609 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5cd28d-8f68-4e1b-84fe-f79b1ba5b16c admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:16.098511 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-1a5cd28d-8f68-4e1b-84fe-f79b1ba5b16c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:47:16.126775 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:47:16.170392 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5cd28d-8f68-4e1b-84fe-f79b1ba5b16c admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:16.174731 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba7dda-01a8-40fa-a04c-752d6d4455fe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d022ed8d2ac43ae816aa396a8e2f17a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:16.175582 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba7dda-01a8-40fa-a04c-752d6d4455fe service glance] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:16.175990 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba7dda-01a8-40fa-a04c-752d6d4455fe service glance] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:16.176374 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba7dda-01a8-40fa-a04c-752d6d4455fe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:16.188451 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba7dda-01a8-40fa-a04c-752d6d4455fe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:16.189944 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba7dda-01a8-40fa-a04c-752d6d4455fe service glance] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:16.200630 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1442/2920] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:47:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:47:26.080772 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:47:26.195896 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=17 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:47:51.355265 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec78d6-93ee-40ec-8b61-feae53cd5f16 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:51.422341 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec78d6-93ee-40ec-8b61-feae53cd5f16 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:51.427128 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390f497-59d3-45d1-a5b5-734550be7642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:51.427780 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390f497-59d3-45d1-a5b5-734550be7642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:51.428104 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390f497-59d3-45d1-a5b5-734550be7642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:51.428384 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0390f497-59d3-45d1-a5b5-734550be7642 admin admin] PATH_INFO: `/v3/users/39b3d9896a34489aa14cbb4210f6315a` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:51.430094 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0390f497-59d3-45d1-a5b5-734550be7642 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39b3d9896a34489aa14cbb4210f6315a)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:51.432064 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0390f497-59d3-45d1-a5b5-734550be7642 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:51.481808 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0390f497-59d3-45d1-a5b5-734550be7642 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39b3d9896a34489aa14cbb4210f6315a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:51.489408 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0390f497-59d3-45d1-a5b5-734550be7642 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39b3d9896a34489aa14cbb4210f6315a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:51.499015 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0390f497-59d3-45d1-a5b5-734550be7642 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39b3d9896a34489aa14cbb4210f6315a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:51.502519 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0390f497-59d3-45d1-a5b5-734550be7642 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39b3d9896a34489aa14cbb4210f6315a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:51.502519 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-0390f497-59d3-45d1-a5b5-734550be7642 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39b3d9896a34489aa14cbb4210f6315a'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:51.528862 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1479/2921] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:47:51 2022] DELETE /identity/v3/users/39b3d9896a34489aa14cbb4210f6315a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:47:51.534436 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91c98f-c82d-46a1-b381-8737478d64a7 None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:51.535018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91c98f-c82d-46a1-b381-8737478d64a7 None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:51.535018 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91c98f-c82d-46a1-b381-8737478d64a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:51.556278 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-0c91c98f-c82d-46a1-b381-8737478d64a7 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:47:51.557547 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0c91c98f-c82d-46a1-b381-8737478d64a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:47:51.620364 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1443/2922] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:47:51.631121 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390f497-59d3-45d1-a5b5-734550be7642 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:51.632641 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-0390f497-59d3-45d1-a5b5-734550be7642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:47:51.706566 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0390f497-59d3-45d1-a5b5-734550be7642 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:51.711341 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf13cf-6473-4bca-af28-78784c1a8a6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:51.712054 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf13cf-6473-4bca-af28-78784c1a8a6b service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:51.712340 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf13cf-6473-4bca-af28-78784c1a8a6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:51.712620 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf13cf-6473-4bca-af28-78784c1a8a6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:51.723735 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf13cf-6473-4bca-af28-78784c1a8a6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:51.725381 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf13cf-6473-4bca-af28-78784c1a8a6b service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:51.737334 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1480/2923] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:47:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:47:52.062639 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91c98f-c82d-46a1-b381-8737478d64a7 None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:52.064326 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-0c91c98f-c82d-46a1-b381-8737478d64a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:47:52.079961 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91c98f-c82d-46a1-b381-8737478d64a7 None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:52.085743 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33e518-0200-4baa-9979-c2b0d9d7f6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:52.086479 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33e518-0200-4baa-9979-c2b0d9d7f6d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:52.087225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33e518-0200-4baa-9979-c2b0d9d7f6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:52.087225 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33e518-0200-4baa-9979-c2b0d9d7f6d3 admin admin] PATH_INFO: `/v3/projects/a1b9811d32784b20941b0ba3178568fb` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:52.088509 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33e518-0200-4baa-9979-c2b0d9d7f6d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1b9811d32784b20941b0ba3178568fb)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:52.090862 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33e518-0200-4baa-9979-c2b0d9d7f6d3 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:52.146515 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ed33e518-0200-4baa-9979-c2b0d9d7f6d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1b9811d32784b20941b0ba3178568fb', 'request_id': 'req-ed33e518-0200-4baa-9979-c2b0d9d7f6d3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:52.152585 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ed33e518-0200-4baa-9979-c2b0d9d7f6d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1b9811d32784b20941b0ba3178568fb', 'request_id': 'req-ed33e518-0200-4baa-9979-c2b0d9d7f6d3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:52.160059 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ed33e518-0200-4baa-9979-c2b0d9d7f6d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1b9811d32784b20941b0ba3178568fb', 'request_id': 'req-ed33e518-0200-4baa-9979-c2b0d9d7f6d3'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:52.172511 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ed33e518-0200-4baa-9979-c2b0d9d7f6d3 admin admin] The token cache is being invalidate because project a1b9811d32784b20941b0ba3178568fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:47:52.172831 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ed33e518-0200-4baa-9979-c2b0d9d7f6d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:52.173491 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-ed33e518-0200-4baa-9979-c2b0d9d7f6d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:47:52.175199 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1444/2924] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:47:52 2022] DELETE /identity/v3/projects/a1b9811d32784b20941b0ba3178568fb => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:47:52.181001 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463141b8-6da1-41e0-9804-7ed56ca59515 None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:52.181292 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463141b8-6da1-41e0-9804-7ed56ca59515 None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:52.181569 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463141b8-6da1-41e0-9804-7ed56ca59515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:52.204009 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-463141b8-6da1-41e0-9804-7ed56ca59515 None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:47:52.205261 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-463141b8-6da1-41e0-9804-7ed56ca59515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:47:52.224811 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:47:52.272965 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1481/2925] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:47:52.284573 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682645a6-4842-46dd-aa4a-10263cd755db None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:52.284882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682645a6-4842-46dd-aa4a-10263cd755db None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:52.284882 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682645a6-4842-46dd-aa4a-10263cd755db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:52.307601 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-682645a6-4842-46dd-aa4a-10263cd755db None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:47:52.309013 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-682645a6-4842-46dd-aa4a-10263cd755db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:47:52.317149 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1445/2926] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:47:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:47:52.321942 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463141b8-6da1-41e0-9804-7ed56ca59515 None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:52.332321 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463141b8-6da1-41e0-9804-7ed56ca59515 None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:52.337092 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443aaa1-d1bf-43fe-bf7d-16254aa74216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:52.337739 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443aaa1-d1bf-43fe-bf7d-16254aa74216 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:52.338025 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443aaa1-d1bf-43fe-bf7d-16254aa74216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:52.338297 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443aaa1-d1bf-43fe-bf7d-16254aa74216 admin admin] PATH_INFO: `/v3/domains` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:52.338985 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443aaa1-d1bf-43fe-bf7d-16254aa74216 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81398) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 29 20:47:52.339298 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443aaa1-d1bf-43fe-bf7d-16254aa74216 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:52.340929 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443aaa1-d1bf-43fe-bf7d-16254aa74216 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:52.354086 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1482/2927] 10.222.0.18 () {58 vars in 1153 bytes} [Mon Aug 29 20:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 29 20:47:52.358761 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682645a6-4842-46dd-aa4a-10263cd755db None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:52.368926 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682645a6-4842-46dd-aa4a-10263cd755db None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:52.373705 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8931d-c8ab-40c4-9cd0-f337f39763f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:52.374319 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8931d-c8ab-40c4-9cd0-f337f39763f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:52.374591 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8931d-c8ab-40c4-9cd0-f337f39763f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:52.374855 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8931d-c8ab-40c4-9cd0-f337f39763f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:52.375477 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8931d-c8ab-40c4-9cd0-f337f39763f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:52.377425 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8931d-c8ab-40c4-9cd0-f337f39763f0 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:52.406686 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1446/2928] 10.222.0.18 () {60 vars in 1153 bytes} [Mon Aug 29 20:47:52 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 29 20:47:52.412867 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443aaa1-d1bf-43fe-bf7d-16254aa74216 admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:52.424721 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443aaa1-d1bf-43fe-bf7d-16254aa74216 admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:52.488419 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1977743-5caf-43ea-9b55-ab1a50d1451f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:52.489008 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1977743-5caf-43ea-9b55-ab1a50d1451f admin admin] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:52.489304 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1977743-5caf-43ea-9b55-ab1a50d1451f admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:52.489784 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1977743-5caf-43ea-9b55-ab1a50d1451f admin admin] PATH_INFO: `/v3/users` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:52.490605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1977743-5caf-43ea-9b55-ab1a50d1451f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:52.492281 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1977743-5caf-43ea-9b55-ab1a50d1451f admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:52.515390 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1483/2929] 10.222.0.18 () {60 vars in 1144 bytes} [Mon Aug 29 20:47:52 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 29 20:47:52.519897 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8931d-c8ab-40c4-9cd0-f337f39763f0 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:52.531723 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8931d-c8ab-40c4-9cd0-f337f39763f0 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:52.536757 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb44176-6755-4a7c-8807-5b84a5fc4cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:52.537412 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb44176-6755-4a7c-8807-5b84a5fc4cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:52.537664 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb44176-6755-4a7c-8807-5b84a5fc4cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:52.537935 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb44176-6755-4a7c-8807-5b84a5fc4cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:52.539157 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb44176-6755-4a7c-8807-5b84a5fc4cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:52.540877 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb44176-6755-4a7c-8807-5b84a5fc4cd5 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:52.551498 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1447/2930] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:47:52.555678 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1977743-5caf-43ea-9b55-ab1a50d1451f admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:52.566103 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1977743-5caf-43ea-9b55-ab1a50d1451f admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:52.571347 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899f0c6-48be-40ef-a96d-3a4bd3efe516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:52.571987 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899f0c6-48be-40ef-a96d-3a4bd3efe516 admin admin] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:52.572274 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899f0c6-48be-40ef-a96d-3a4bd3efe516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:52.572551 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899f0c6-48be-40ef-a96d-3a4bd3efe516 admin admin] PATH_INFO: `/v3/roles` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:52.573231 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7899f0c6-48be-40ef-a96d-3a4bd3efe516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:52.574825 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7899f0c6-48be-40ef-a96d-3a4bd3efe516 admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:52.586770 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1484/2931] 10.222.0.18 () {58 vars in 1122 bytes} [Mon Aug 29 20:47:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 29 20:47:52.591264 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb44176-6755-4a7c-8807-5b84a5fc4cd5 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:52.602256 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb44176-6755-4a7c-8807-5b84a5fc4cd5 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:52.606999 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2b26f-b944-43cb-99d3-a683827ee0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:52.607661 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2b26f-b944-43cb-99d3-a683827ee0d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:52.607957 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2b26f-b944-43cb-99d3-a683827ee0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:52.608249 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2b26f-b944-43cb-99d3-a683827ee0d1 admin admin] PATH_INFO: `/v3/projects/a40ba812002c41bf9eff91bfa527139c/users/6963ec79963f468d9540e8856f045816/roles/89cf5ed93cc642588a54639342c59ea6` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:52.623511 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2b26f-b944-43cb-99d3-a683827ee0d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a40ba812002c41bf9eff91bfa527139c, user_id=6963ec79963f468d9540e8856f045816, role_id=89cf5ed93cc642588a54639342c59ea6)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:52.626240 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2b26f-b944-43cb-99d3-a683827ee0d1 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:52.640611 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1448/2932] 10.222.0.18 () {60 vars in 1483 bytes} [Mon Aug 29 20:47:52 2022] PUT /identity/v3/projects/a40ba812002c41bf9eff91bfa527139c/users/6963ec79963f468d9540e8856f045816/roles/89cf5ed93cc642588a54639342c59ea6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:47:52.651708 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47cb07-8cdd-4d2a-b450-0cec6ed0366d None None] REQUEST_METHOD: `POST` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:52.652004 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47cb07-8cdd-4d2a-b450-0cec6ed0366d None None] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:52.652283 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47cb07-8cdd-4d2a-b450-0cec6ed0366d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:52.690555 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.auth.core [None req-ac47cb07-8cdd-4d2a-b450-0cec6ed0366d None None] MFA Rules not processed for user `6963ec79963f468d9540e8856f045816`. Rule list: `[]` (Enabled: `True`). {{(pid=81398) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:47:52.696337 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-ac47cb07-8cdd-4d2a-b450-0cec6ed0366d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:47:52.779531 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1485/2933] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:47:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:47:52.793146 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2b26f-b944-43cb-99d3-a683827ee0d1 admin admin] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:47:52.795113 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-bba2b26f-b944-43cb-99d3-a683827ee0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:47:52.822538 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:47:52.870427 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2b26f-b944-43cb-99d3-a683827ee0d1 admin admin] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:47:52.874511 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96e0cd-2b1a-49cb-9700-055932b5f1a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85624cea63af40fda09f57ddf2cd5c3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:47:52.875223 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96e0cd-2b1a-49cb-9700-055932b5f1a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:47:52.875535 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96e0cd-2b1a-49cb-9700-055932b5f1a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:47:52.875875 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96e0cd-2b1a-49cb-9700-055932b5f1a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:47:52.887636 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f96e0cd-2b1a-49cb-9700-055932b5f1a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:47:52.889112 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f96e0cd-2b1a-49cb-9700-055932b5f1a6 service cinder] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:47:52.899373 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1449/2934] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:47:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:48:02.776001 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:48:02.894548 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=17 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:48:12.469071 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47cb07-8cdd-4d2a-b450-0cec6ed0366d None None] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:48:12.533429 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47cb07-8cdd-4d2a-b450-0cec6ed0366d None None] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:48:12.537859 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db3ddb-c556-4f85-81ab-3d65ee91eeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:48:12.538511 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db3ddb-c556-4f85-81ab-3d65ee91eeab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:48:12.538797 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db3ddb-c556-4f85-81ab-3d65ee91eeab admin admin] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:48:12.539069 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db3ddb-c556-4f85-81ab-3d65ee91eeab admin admin] PATH_INFO: `/v3/users/6963ec79963f468d9540e8856f045816` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:48:12.540224 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db3ddb-c556-4f85-81ab-3d65ee91eeab admin admin] RBAC: Authorizing `identity:delete_user(user_id=6963ec79963f468d9540e8856f045816)` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:48:12.542236 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db3ddb-c556-4f85-81ab-3d65ee91eeab admin admin] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:48:12.599193 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f1db3ddb-c556-4f85-81ab-3d65ee91eeab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6963ec79963f468d9540e8856f045816'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:48:12.606433 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f1db3ddb-c556-4f85-81ab-3d65ee91eeab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6963ec79963f468d9540e8856f045816'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:48:12.614445 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f1db3ddb-c556-4f85-81ab-3d65ee91eeab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6963ec79963f468d9540e8856f045816'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:48:12.615071 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f1db3ddb-c556-4f85-81ab-3d65ee91eeab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6963ec79963f468d9540e8856f045816'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:48:12.616223 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.notifications [None req-f1db3ddb-c556-4f85-81ab-3d65ee91eeab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6963ec79963f468d9540e8856f045816'} {{(pid=81398) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:48:12.637726 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1486/2935] 10.222.0.18 () {58 vars in 1224 bytes} [Mon Aug 29 20:48:12 2022] DELETE /identity/v3/users/6963ec79963f468d9540e8856f045816 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 29 20:48:12.642447 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f118b19-dbe2-4b8c-8515-9434fee57caa None None] REQUEST_METHOD: `POST` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:48:12.642979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f118b19-dbe2-4b8c-8515-9434fee57caa None None] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:48:12.642979 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f118b19-dbe2-4b8c-8515-9434fee57caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:48:12.663327 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.auth.core [None req-7f118b19-dbe2-4b8c-8515-9434fee57caa None None] MFA Rules not processed for user `b4cb9e8f0d014047a2ac687139b6261f`. Rule list: `[]` (Enabled: `True`). {{(pid=81397) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 29 20:48:12.664594 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7f118b19-dbe2-4b8c-8515-9434fee57caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:48:12.733445 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1450/2936] 10.222.0.18 () {58 vars in 958 bytes} [Mon Aug 29 20:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 29 20:48:12.744209 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db3ddb-c556-4f85-81ab-3d65ee91eeab admin admin] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:48:12.745634 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-f1db3ddb-c556-4f85-81ab-3d65ee91eeab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:48:12.805184 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db3ddb-c556-4f85-81ab-3d65ee91eeab admin admin] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:48:12.808864 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fde4a6-9fff-4b70-9066-2398c059f84a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd51e331a4c9c4f038bdb80f8c5b74e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:48:12.809501 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fde4a6-9fff-4b70-9066-2398c059f84a service neutron] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:48:12.809791 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fde4a6-9fff-4b70-9066-2398c059f84a service neutron] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:48:12.810072 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fde4a6-9fff-4b70-9066-2398c059f84a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:48:12.822215 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fde4a6-9fff-4b70-9066-2398c059f84a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:48:12.824605 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fde4a6-9fff-4b70-9066-2398c059f84a service neutron] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:48:12.839106 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1487/2937] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:48:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:48:13.132639 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f118b19-dbe2-4b8c-8515-9434fee57caa None None] Authenticating user token {{(pid=81397) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:48:13.134495 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.fernet_utils [None req-7f118b19-dbe2-4b8c-8515-9434fee57caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81397) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:48:13.152265 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f118b19-dbe2-4b8c-8515-9434fee57caa None None] Validating token access rules against request {{(pid=81397) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:48:13.158269 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6bc69-b595-4ee1-b898-393f94338fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4cb9e8f0d014047a2ac687139b6261f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31796c851dfa4e61a8e1afef74d31165', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81397) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:48:13.159322 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6bc69-b595-4ee1-b898-393f94338fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:48:13.159595 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6bc69-b595-4ee1-b898-393f94338fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:48:13.160053 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6bc69-b595-4ee1-b898-393f94338fd1 admin admin] PATH_INFO: `/v3/projects/a40ba812002c41bf9eff91bfa527139c` {{(pid=81397) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:48:13.161164 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6bc69-b595-4ee1-b898-393f94338fd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a40ba812002c41bf9eff91bfa527139c)` {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:48:13.163700 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6bc69-b595-4ee1-b898-393f94338fd1 admin admin] RBAC: Authorization granted {{(pid=81397) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:48:13.231272 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-efd6bc69-b595-4ee1-b898-393f94338fd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a40ba812002c41bf9eff91bfa527139c', 'request_id': 'req-efd6bc69-b595-4ee1-b898-393f94338fd1'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:48:13.237369 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-efd6bc69-b595-4ee1-b898-393f94338fd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a40ba812002c41bf9eff91bfa527139c', 'request_id': 'req-efd6bc69-b595-4ee1-b898-393f94338fd1'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:48:13.245944 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-efd6bc69-b595-4ee1-b898-393f94338fd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a40ba812002c41bf9eff91bfa527139c', 'request_id': 'req-efd6bc69-b595-4ee1-b898-393f94338fd1'} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:48:13.261501 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-efd6bc69-b595-4ee1-b898-393f94338fd1 admin admin] The token cache is being invalidate because project a40ba812002c41bf9eff91bfa527139c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81397) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 29 20:48:13.261812 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-efd6bc69-b595-4ee1-b898-393f94338fd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:48:13.262718 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG keystone.notifications [None req-efd6bc69-b595-4ee1-b898-393f94338fd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81397) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 29 20:48:13.264457 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: [pid: 81397|app: 0|req: 1451/2938] 10.222.0.18 () {58 vars in 1233 bytes} [Mon Aug 29 20:48:13 2022] DELETE /identity/v3/projects/a40ba812002c41bf9eff91bfa527139c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 29 20:48:22.833238 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:48:23.256642 dv-cinder-854263-2-iscsi devstack@keystone.service[81397]: DEBUG dbcounter [-] [81397] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81397) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 29 20:51:10.960479 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fde4a6-9fff-4b70-9066-2398c059f84a service neutron] Authenticating user token {{(pid=81398) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 29 20:51:10.961892 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-10fde4a6-9fff-4b70-9066-2398c059f84a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:51:11.113284 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fde4a6-9fff-4b70-9066-2398c059f84a service neutron] Validating token access rules against request {{(pid=81398) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 29 20:51:11.179911 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb2752-7c33-4c45-ae98-4799c159b9b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fa0e659eee5e4a9da8d3d7337cb73798', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90b25dc95d5a4a068994bdd801d6b8ea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81398) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 29 20:51:11.180557 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb2752-7c33-4c45-ae98-4799c159b9b6 service placement] REQUEST_METHOD: `GET` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 29 20:51:11.180837 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb2752-7c33-4c45-ae98-4799c159b9b6 service placement] SCRIPT_NAME: `/identity` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 29 20:51:11.181110 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb2752-7c33-4c45-ae98-4799c159b9b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81398) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 29 20:51:11.182849 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.fernet_utils [None req-4bfb2752-7c33-4c45-ae98-4799c159b9b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81398) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 29 20:51:11.195744 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb2752-7c33-4c45-ae98-4799c159b9b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 29 20:51:11.197380 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb2752-7c33-4c45-ae98-4799c159b9b6 service placement] RBAC: Authorization granted {{(pid=81398) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 29 20:51:11.207494 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: [pid: 81398|app: 0|req: 1488/2939] 10.222.0.18 () {60 vars in 1365 bytes} [Mon Aug 29 20:51:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 29 20:51:21.203125 dv-cinder-854263-2-iscsi devstack@keystone.service[81398]: DEBUG dbcounter [-] [81398] Writing DB stats keystone:SELECT=89 {{(pid=81398) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}